Interactive biometric touch scanner

ABSTRACT

Aspects of this disclosure relate to a biometric sensing device that combines sensing with an actuator for two way communication between a finger on a surface and the device. The sensor can also function as an actuator. A finger can be authenticated based on an image of the finger generated by the sensor and also based on a response to energy delivered to the finger by the actuator. Two way communication can provide more robust authentication than fingerprint sensing alone.

CROSS REFERENCE TO PRIORITY APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/057,666, filed Aug. 7, 2018, entitled “INTERACTIVE BIOMETRIC TOUCHSCANNER,” which claims the benefit of priority of U.S. ProvisionalPatent Application No. 62/543,280, filed Aug. 9, 2017, entitled“BIOMETRIC TOUCH SCANNER INTEGRATED WITH OPTICS,” and of U.S.Provisional Patent Application No. 62/543,278, filed Aug. 9, 2017,entitled “INTERACTIVE BIOMETRIC TOUCH SCANNER.” The contents of each ofthe above-mentioned applications are hereby incorporated by referenceherein in their entireties and for all purposes.

BACKGROUND Technological Field

The disclosed technology relates to biometric scanning, includingapplications to fingerprint recognition and live finger detection.

Description of the Related Technology

Fingerprints have been associated with a wide variety of applicationsand uses including criminal identification, banking, ID recognition forpersonal devices, official forms, and others. Automated opticalfingerprint scanners have been used to acquire fingerprint images.Ultrasound-based fingerprint scanners and capacitive fingerprintscanners are other fingerprint detection technologies. There is a needfor robust and cost-effective fingerprint scanning systems with robustauthentication.

SUMMARY OF CERTAIN INVENTIVE ASPECTS

The innovations described in the claims each have several aspects, nosingle one of which is solely responsible for its desirable attributes.Without limiting the scope of the claims, some prominent features ofthis disclosure will now be briefly described.

One aspect of the disclosed technology is a biometric sensing device.The device includes a surface configured to receive a finger. The devicefurther includes an ultrasonic fingerprint sensor comprising ultrasonictransducers configured to transmit an ultrasound signal to the surface.The ultrasonic fingerprint sensor is configured to generate dataindicative of an image of at least a portion of a fingerprint of thefinger on the surface. The device further includes an optical systemintegrated with the fingerprint sensor. The optical system is configuredto transmit light to the receiving surface through the ultrasonicfingerprint sensor.

In an embodiment, the ultrasonic transducers are transparent to thelight transmitted by the optical system. In an embodiment, theultrasonic fingerprint sensor includes electrodes for addressing theultrasonic transducers in which the electrodes are transparent to thelight transmitted by the optical system.

In an embodiment, the ultrasonic transducers are positioned under thereceiving surface and the optical system is positioned under theultrasonic transducers.

In an embodiment, the ultrasonic transducers are positioned under thereceiving surface and the optical system includes a light source and anoptical sensor that are positioned laterally relative to the ultrasonictransducers.

In an embodiment, the ultrasonic transducers are positioned between thereceiving surface and the optical system.

In an embodiment, the ultrasound signal has a frequency in a range from50 megahertz to 500 megahertz. In an embodiment, the optical systemincludes a reflective pulse oximeter. In an embodiment, the opticalsystem is configured to transmit light at two or more differentwavelengths.

In an embodiment, the device further includes a processor. In anembodiment, the processor is configured to generate a liveness parameterbased on a comparison of the light at the two or more differentwavelengths reflected by the finger. In an embodiment, the processor isconfigured to generate a liveness parameter based on light reflected bythe finger that is received by the optical system. In an embodiment, theliveness parameter is indicative of at least one of a heart rate, ablood oxygenation level, or a temperature. In an embodiment, theprocessor is configured to output an indication of whether the finger isalive.

In an embodiment, the device further includes a layer of transparentmaterial, such as a glass or plastic layer, positioned between thefingerprint sensor and the surface configured to receive a finger. In anembodiment, the surface configured to receive a finger is a surface ofthe layer of transparent material.

Another aspect is a biometric sensing device. The device includesultrasonic transducers configured to transmit an ultrasound signal to anobject. The device further includes an optical system integrated withthe ultrasonic transducers. The optical system is configured to transmitlight to the object and receive light reflected from the object. Thedevice further includes one or more processors. The one or moreprocessors are configured to generate an image of at least a portion ofthe object based on a reflection of the ultrasound signal from theobject. The one or more processors are further configured to generate aliveness parameter based on the received light reflected from theobject.

In an embodiment, the optical system is configured to transmit the lightthrough the ultrasonic transducers to the object.

In an embodiment, the device further includes electrodes for addressingthe ultrasonic transducers, and the electrodes are transparent to thelight transmitted by the optical system. In an embodiment, theelectrodes and the ultrasonic transducers are both transparent to thelight transmitted by the optical system.

In an embodiment, the device further includes electrodes for addressingthe ultrasonic transducers. In an embodiment, the electrodes are opaqueto the light transmitted by the optical system.

In an embodiment, the ultrasonic transducers are positioned under thereceiving surface and the optical system is positioned under theultrasonic transducers. In an embodiment, the ultrasonic transducers arepositioned under the receiving surface and the optical system includes alight source and an optical sensor that are positioned laterallyrelative to the ultrasonic transducers.

In an embodiment, the ultrasonic transducers are transparent to thelight transmitted by the optical system. In an embodiment, theultrasonic transducers are arranged as an array. In an embodiment, theoptical system comprises a light source and a light sensor that areintegrated within the array.

In an embodiment, the ultrasound signal has a frequency in a range from50 megahertz to 500 megahertz. In an embodiment, the device furtherincludes a surface configured to receive the object and a glass layerpositioned between the ultrasonic transducers and the surface.

Another aspect is biometric sensing device. The device includes a sensorconfigured to generate data indicative of an image of at least a portionof an object. The device further includes an optical system integratedwith the sensor, the optical system configured to transmit light to theobject through the sensor. In certain embodiments, the object includes afinger, a palm, a sole of a foot, a toe, etc.

Another aspect is a biometric sensing device. The device includesultrasonic transducers configured to transmit an ultrasound signal to anobject. The device further includes an optical system integrated withthe ultrasonic transducers. The optical system is configured to transmitlight to the object and receive light reflected from the object. Thedevice further includes one or more processors. The one or moreprocessors are configured to generate an image of at least a portion ofthe object based on a reflection of the ultrasound signal from theobject and to generate a liveness parameter based on the received light.In an embodiment, the object includes a finger.

Another aspect is a method of biometric authentication. The methodincludes transmitting, by a fingerprint sensor comprising apiezoelectric layer, an ultrasound signal to a finger. The methodfurther includes generating an image of at least a portion of the fingerbased on a reflection of the ultrasound signal from the finger. Themethod further includes transmitting light through the piezoelectriclayer of the fingerprint sensor to the finger. The method furtherincludes generating a liveness parameter based on a reflection of thelight from the finger. The method further includes authenticating a userbased on the image and the liveness parameter.

In an embodiment, the signal is an ultrasound signal and the receivedsignal is a reflection of the ultrasound signal from the finger. In anembodiment, the signal is a light signal and the received signal is areflection of the light signal from the finger. The ultrasonic signalcan be transmitted through glass. The method can be performed by amobile phone that comprises the fingerprint sensor and an optical systemconfigured to transmit the light. The method can be performed using asmart card that includes the fingerprint sensor.

Another aspect is an interactive biometric sensing system. The systemincludes a sensor configured to generate a biometric image associatedwith an object. The system further includes an actuator configured todeliver energy to the object. The system further includes a processorconfigured to authenticate the object based on the biometric image and aresponse to the energy delivered by the actuator.

In an embodiment, the sensor is configured to implement the actuator. Inan embodiment, the actuator is configured to detect the response andprovide an indication of the response to the processor.

In an embodiment, the actuator is part of a computing device thatincludes the fingerprint sensor. For example, the actuator can includeMEMS devices of a mobile phone that includes the interactive biometricsystem. In this example, the MEMS devices can also be arranged to makethe mobile phone vibrate.

In an embodiment, the interactive biometric sensing system is configuredto detect a real-time response to the energy delivered to the object.

In an embodiment, the object is a finger. In an embodiment, thebiometric image is an image of a fingerprint.

In an embodiment, the system further includes a surface configured toreceive the object. In an embodiment, the actuator is configured todeliver the energy to the object while the object is on the surface.

In an embodiment, the system further includes a surface configured toreceive the object, and the response to the energy delivered to theobject is associated with the object being on the receiving surface.

In an embodiment, the response is involuntary. In an embodiment, theresponse is voluntary.

In an embodiment, the actuator is configured to cause a change in atemperature of the object, and the response is a change in thetemperature of the object. In an embodiment, the actuator is configuredto apply pressure to the object.

In an embodiment, the sensor includes ultrasound transducers. In anembodiment, the actuator includes the ultrasound transducers.

In an embodiment, the ultrasound transducers are configured to applypressure to the object and the response is to the pressure applied tothe object.

In an embodiment, the ultrasound transducers are configured to cause achange in a temperature of the object, and the response is detectedusing the ultrasound transducers.

In an embodiment, the actuator includes a light source configured totransmit light to the object.

In an embodiment, the actuator includes a temperature sensor configuredto cause a change in a temperature of the object. In an embodiment, thesensor includes a capacitive sensor. In an embodiment, the sensorcomprises an optical system.

Another aspect of this disclosure is an interactive biometricauthentication system comprising: a sensor and a processor. The sensoris configured to generate biometric image data associated with anobject. The sensor is further configured to deliver energy to theobject. The processor is in communication with the sensor. The processoris configured to authenticate the object based on the biometric imagedata and an indication of a response to the energy delivered to theobject by the sensor.

The sensor can include ultrasound transducers. The ultrasoundtransducers can be configured to apply pressure to the object, and theresponse is to the pressure applied to the object. The ultrasoundtransducers can be configured to cause a change in a temperature. Theresponse can be detected using the ultrasound transducers.

The sensor can be configured to detect the response and provide theindication of the response to the processor. The sensor can beconfigured to deliver the energy to the object as a prompt in a mannerthat exhibits statistical randomness.

The interactive biometric authentication system can be configured todetect a real-time response to the energy delivered to the object. Theprocessor can be configured to authenticate the object on a millisecondtimescale after the energy is delivered to the object.

The interactive biometric authentication system can further include asurface configured to receive the object, wherein the sensor isconfigured to deliver energy to the object while the object ispositioned on the surface. The interactive biometric authenticationsystem can further include engineered glass disposed between the sensorand the surface, wherein the sensor is configured to deliver the energyto the object through the engineered glass.

The response can be involuntary. Alternatively, the response can bevoluntary.

The interactive biometric authentication system can further include auser interface configured to receive the response.

Another aspect of this disclosure is method of interactivelyauthenticating a person. The method comprises: transmitting, by afingerprint sensor, a signal to a finger of the person positioned on asurface; generating an image of at least a portion of the finger basedon a received signal associated with the signal transmitted to thefinger; delivering energy to the finger while the finger is positionedon the surface; detecting a response to the energy delivered to thefinger; and authenticating the person based on the image and thedetecting.

The fingerprint sensor can include ultrasound transducers, and thedelivering is performed using the ultrasound transducers. The detectingcan include detecting the response via a user interface. The method canbe performed by a mobile phone.

Another aspect of this disclosure is a mobile phone with interactivebiometric authentication. The mobile phone comprises: an antennaconfigured to a transmit a wireless communication signal; a surfaceconfigured to receive a finger; a sensor configured to generatebiometric image data associated with the finger being positioned on thesurface, the sensor being further configured to deliver energy to thefinger positioned on the surface; and a processor in communication withthe sensor, the processor configured to authenticate the finger based onthe biometric image data and an indication of a response to the energydelivered to the finger by the sensor.

The sensor can include ultrasound transducers. The mobile phone canfurther include engineered glass disposed between the sensor and thesurface.

Another aspect is an interactive biometric sensing device. The deviceincludes a surface configured to receive an object. The device furtherincludes a sensor configured to generate biometric informationassociated with the object, deliver energy to the object while theobject is on the surface, and detect a response to the delivered energy.

Another aspect is a method of authenticating a user. The method includestransmitting, by a fingerprint sensor, a signal to a receiving surface.The method further includes generating an image of at least a portion ofa finger on the receiving surface based on a received signal associatedwith the signal transmitted to the finger. The method further includesdelivering energy to the finger. The method further includes generatinga liveness parameter based on a detected response to the energydelivered to the finger. The method further includes authenticating auser based on the image and the liveness parameter.

For purposes of summarizing the disclosure, certain aspects, advantagesand novel features of the innovations have been described herein. It isto be understood that not necessarily all such advantages may beachieved in accordance with any particular embodiment. Thus, theinnovations may be embodied or carried out in a manner that achieves oroptimizes one advantage or group of advantages as taught herein withoutnecessarily achieving other advantages as may be taught or suggestedherein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates acoustic fingerprint scanning, in which an ultrasoundtransducer emits an ultrasound wave which can be strongly reflected andweekly transmitted at the surface-finger interface and also from withinthe finger as shown.

FIG. 2 illustrates a device for focusing sound waves with a row-columnaddressed two-dimensional (2D) array. Only one transmit focal line canbe active at a time. One or more receive focus lines can be active at atime. The transmit and receive focus lines are perpendicular to eachother and intersect at a measurement spot with a compact focal spotsize.

FIG. 3 illustrates a two-dimensional row-column addressed array oftransducer elements addressed by a vertical array of row electrodes anda horizontal array of column electrodes, the vertical and horizontalarrays orthogonal to each other and on different sides of the array.

FIG. 4 illustrates a perspective view of an example ultrasoundtransducer array mounted on a substrate.

FIG. 5 illustrates a perspective view of a portion of an ultrasoundtransducer array mounted on a substrate.

FIG. 6 illustrates an intermediate step of manufacturing an ultrasoundtransducer array by depositing bottom electrodes on top of a glasssubstrate.

FIG. 7 illustrates an intermediate step of manufacturing an ultrasoundtransducer array by depositing piezoelectric film over the bottomelectrodes.

FIG. 8 illustrates an intermediate step of manufacturing an ultrasoundtransducer array by etching trenches or grooves in two directions on thetop side of the film to reduce crosstalk between elements.

FIG. 9 illustrates an intermediate step of manufacturing an ultrasoundtransducer array by depositing top electrodes in a perpendiculardirection relative to the bottom electrodes.

FIG. 10 illustrates an example acoustic biometric touch scanner,including an ultrasound transducer array, transmit electronics, andreceive electronics.

FIG. 11 illustrates a multiplexed single channel row-column addressedtransducer array with an intersection of a single active row and asingle active column.

FIG. 12 illustrates a multiplexed single channel row-column addressedtransducer array with an intersection of three rows and two columns.

FIG. 13 illustrates peak detection circuitry using op-amps to detect apeak in an ultrasound signal.

FIG. 14 illustrates frequency domain plots associated with signals inreceive circuitry in communication with an ultrasound transducer array.

FIG. 15 illustrates a functional block diagram for direct in-phase andquadrature (IQ) sampling in receive circuitry in communication with anultrasound transducer array.

FIG. 16 illustrates an original high frequency signal and its spectralaliases when undersampled, and the baseband alias after undersampling.

FIG. 17 illustrates that as the force with which a finger is pushedagainst a receive surface increases, fingerprint ridges widen and thetotal finger surface in contact with the receiving surface increases.

FIG. 18 illustrates the speed of a sound wave through a medium canchange with a change in temperature in the medium.

FIG. 19 illustrates that the time of flight from excitation until thereflected wavefront is recorded can change with temperature.

FIG. 20 is a flowchart of a method of generating biometric informationaccording to an embodiment of the disclosed technology.

FIG. 21 is a flowchart of a method of generating a biometric imageaccording to an embodiment of the disclosed technology.

FIG. 22 is a flowchart of a method of manufacturing an acousticbiometric touch scanner according to an embodiment of the disclosedtechnology.

FIG. 23 is a flowchart of a method of detecting a temperature of afinger according to an embodiment of the disclosed technology.

FIG. 24 is a flowchart of a method of estimating a force at which afinger contacts a surface according to an embodiment of the disclosedtechnology.

FIG. 25 is a flowchart of a method of estimating period of a time seriesof force measurements, the period corresponding to a pulse rateestimate, according to an embodiment of the disclosed technology.

FIGS. 26-34 illustrates circuits and results of simulations of samplingand envelope detection methods for ultrasound finger print scanning.

FIG. 26 illustrates a simulation of the one-way insertion loss.

FIG. 27 illustrates a circuit for IQ demodulation of an RF signal into Iand Q channels.

FIG. 28 illustrates an example of the simulated demodulated in-phase andquadrature signals.

FIG. 29 illustrates the response of the low pass filter used for IQdemodulation for the process of FIG. 27.

FIG. 30 illustrates an IQ demodulated envelope for a signal demodulatedby the circuit of FIG. 27.

FIG. 31 illustrates 100 MHz samples taken of the IQ demodulated envelopeof FIG. 30.

FIG. 32 illustrates a circuit for IQ sampling of an IQ demodulatedsignal.

FIG. 33 illustrates sampled in-phase and quadrature signals of an IQdemodulated signal.

FIG. 34 illustrates graphs of the envelope of an IQ demodulated signalfor IQ sampling rates of 200 MHz, 150 MHz, 100 MHz and 50 MHz.

FIGS. 35-45 illustrate an example embodiment with an optical systembelow an ultrasound transducer array with transparent metal electrodes.The ultrasound transducer array is below the glass and a receivingsurface for a finger or other object to be examined.

FIG. 35 illustrates an ultrasound transducer array with transparent topand bottom metal electrodes.

FIG. 36 illustrates an exploded view of the ultrasound transducer arrayof FIG. 35 above an optical system and below glass, with the glass,ultrasound transducer array and optical system.

FIG. 37 illustrates the integration of the optical system, ultrasoundtransducer array, and glass of FIG. 36. Unlike FIG. 36, the componentsare illustrated in close proximity to each other. They can adjoin andnot be spatially separated.

FIG. 38 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during transmission of light at a firstwavelength from a light emitter of the optical system through thetransparent transducer array and glass to a finger on the receivingsurface of the glass.

FIG. 39 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during reception of reflected light at thefirst wavelength off of the finger through the glass and the transparenttransducer array to an optical sensor in the optical system.

FIG. 40 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during transmission of light at a secondwavelength from a light emitter of the optical system through thetransparent transducer array and glass to a finger on the receivingsurface of the glass.

FIG. 41 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during reception of reflected light at thesecond wavelength off of the finger through the glass and thetransparent transducer array to an optical sensor in the optical system.

FIG. 42 illustrates the light transmission and reception at the firstand second wavelengths, as illustrated in FIGS. 38-41. Comparisons inreceived light at different wavelengths may be used to, for example,take a reflected pulse oximetry or any other suitable reading of afinger on the receiving surface.

FIG. 43 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with transparent metalelectrodes during a transmit phase without a finger on the receivingsurface.

FIG. 44 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with transparent metalelectrodes during a transmit phase with a finger on the receivingsurface.

FIG. 45 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with transparent metalelectrodes during a receive phase with a finger on the receivingsurface.

FIGS. 46-55 illustrate an example embodiment with an optical systembelow an ultrasound transducer array with opaque metal electrodes. Theultrasound transducer is below glass and a receiving surface for afinger or other object to be examined.

FIG. 46 illustrates an ultrasound transducer array with opaque top andbottom metal electrodes.

FIG. 47 illustrates an exploded view of the ultrasound transducer arrayof FIG. 46 above an optical system and below glass, with the glass,ultrasound transducer array and optical system. As shown in FIGS. 48-55,the glass, ultrasound transducer array, and optical system can be inclose proximity to each other. They can adjoin and not be spatiallyseparated.

FIG. 48 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 47 during transmission of light at a firstwavelength from a light emitter of the optical system through thetransparent transducer array and glass to a finger with on the receivingsurface of the glass. FIG. 48 illustrates that the transparenttransducer array is transparent between the opaque metal electrodes.

FIG. 49 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thefirst wavelength off of the finger through the glass and the transparenttransducer array to an optical sensor in the optical system. Thetransparent transducer array is transparent between the opaque metalelectrodes.

FIG. 50 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 47 during transmission of light at a secondwavelength from a light emitter of the optical system through thetransparent transducer array and glass to a finger on the receivingsurface of the glass. FIG. 48 illustrates that the transparenttransducer array is transparent between the opaque metal electrodes.

FIG. 51 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thesecond wavelength off of the finger through the glass and thetransparent transducer array to an optical sensor in the optical system.The transparent transducer array is transparent between the opaque metalelectrodes.

FIG. 52 illustrates the light transmission and reception at the firstand second wavelengths, as illustrated in FIGS. 48-51. Comparisons inreceived light at different wavelengths may be used to, for example,take a reflected pulse oximetry reading of a finger on the receivingsurface.

FIG. 53 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with opaque metal electrodesduring a transmit phase without a finger on the receiving surface.

FIG. 54 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with opaque metal electrodesduring a transmit phase with a finger on the receiving surface.

FIG. 55 is a perspective view of the example embodiment with an opticalsystem below an ultrasound transducer array with opaque metal electrodesduring a receive phase with a finger on the receiving surface.

FIGS. 56-65 illustrate an example embodiment with an optical systemintegrated inside an array of ultrasound transducers.

FIG. 56 illustrates an ultrasound transducer array with opaque top andbottom metal electrodes, with integrated light sources and lightsensors.

FIG. 57 illustrates an exploded view of the ultrasound transducer arraywith integrated light sources and light sensors of FIG. 56 below glass,with the glass, and ultrasound transducer array with integrated lightsources and sensors. As shown in FIGS. 58-66, the glass and ultrasoundtransducer array with integrated light sources and sensors can be inclose proximity to each other. They can be adjoined and not spatiallyseparated.

FIG. 58 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during transmission of light at a firstwavelength from a light emitter through the glass to a finger on thereceiving surface of the glass.

FIG. 59 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thefirst wavelength off of the finger through the glass to an opticalsensor.

FIG. 60 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during transmission of light at a secondwavelength from a light emitter through the glass to a finger on thereceiving surface of the glass.

FIG. 61 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thefirst wavelength off of the finger through the glass to an opticalsensor.

FIG. 62 illustrates the light transmission and reception at the firstand second wavelengths, as illustrated in FIGS. 58-61. Comparisons inreceived light at different wavelengths may be used to, for example,take a reflected pulse oximetry reading of a finger on the receivingsurface.

FIG. 63 is a perspective view of the example embodiment with an opticalsystem integrated inside the ultrasound transducer array with opaquemetal electrodes during a transmit phase without a finger on thereceiving surface.

FIG. 64 is a perspective view of the example embodiment with an opticalsystem integrated inside the ultrasound transducer array with opaquemetal electrodes with opaque metal electrodes during a transmit phasewith a finger on the receiving surface.

FIG. 65 is a perspective view of the example embodiment with an opticalsystem integrated inside the ultrasound transducer array with opaquemetal electrodes with opaque metal electrodes during a receive phasewith a finger on the receiving surface.

FIG. 66 illustrates an example embodiment with an optical system nextto/adjoining an ultrasound transducer array with opaque metalelectrodes. FIG. 66 illustrates a transmit phase of this embodiment.

FIG. 67 illustrates an example acoustic biometric touch scanner,including an ultrasound system and an optical system. The ultrasoundsystem includes an ultrasound transducer array, transmit electronics,and receive electronics. The optical system includes a light source, anoptical sensor, and supporting electronics.

FIG. 68 illustrates an example biometric touch scanner, including anultrasound system and an optical system. The ultrasound system andoptical system share control a processor and control circuitry.

FIG. 69 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a light source actuator below anultrasound transducer array.

FIG. 70 illustrates the embodiment of FIG. 69, for which the lightsource actuator shines light through the ultrasound transducer array andglass to a finger on the receiving surface of the glass, such that thelight source incrementally heats the finger.

FIG. 71 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a point focus ultrasound heaterthat focuses ultrasound from the ultrasound transducer array throughglass to a point (small region) of a finger on the receiving surface ofthe glass.

FIG. 72 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a line focus ultrasound heaterthat focuses ultrasound from the ultrasound transducer array throughglass to a line (line segment) of a finger on the receiving surface ofthe glass.

FIG. 73 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a resistance based heater capableof sending current through the electrodes (top and bottom metalelectrodes) of the ultrasound transducer array.

FIG. 74 illustrates an example embodiment of FIG. 73, of a biometrictouch scanner with two way communication, including a resistance basedheater capable of sending current through the electrodes (top and bottommetal electrodes) of the ultrasound transducer array, emanating heatfrom the ultrasound transducer array through glass to a finger on thereceiving surface of the glass.

FIG. 75 illustrates operation of the example embodiment of FIGS. 73 and74, of a biometric touch scanner with two-way communication, including aresistance based heater capable of sending current through theelectrodes (top and bottom metal electrodes) of the ultrasoundtransducer array.

FIGS. 76-81 illustrate representative steps of two-way communicationscenarios. The first scenario is illustrated in FIGS. 76-79. The secondscenario is illustrated in FIGS. 76, 77, 80 and 81.

FIG. 76 illustrates the user interface of a representative portablecommunications device including an acoustic biometric touch scanner anda display for measurement or indications of heart rate, pulse oxidationlevels, blood flow, temperature, two way authentication, and fingerprintdetection.

FIG. 77 illustrates an intermediate step of the two-way communicationscenarios of FIGS. 76-81, in which the user's fingerprint is scanned andbiometric information acquired.

FIG. 78 illustrates an intermediate step of the two-way communicationscenario of FIGS. 76-79. After scanning the biometric measures, thedevice generates a sensation at the user's fingertip with an actuator.The user is then prompted to input what sensation is felt. In FIG. 78, asensation corresponding to shape A is drawn on the user's fingertip.

FIG. 79 illustrates an intermediate step of the two way communicationscenario of FIGS. 76-79. The user is prompted to enter the letter of theshape sensed at the fingertip. If the user enters the shape that wasdrawn, the user is authenticated.

FIG. 80 illustrates an intermediate step of the two way communicationscenario of FIGS. 76, 77, 80 and 81. After scanning the biometricmeasures, the device generates a sensation at the user's fingertip. Theuser is then prompted to input what sensation is felt. In FIG. 80, asensation corresponding to three pulses is applied to the user'sfingertip.

FIG. 81 illustrates an intermediate step of the two way communicationscenario of FIGS. 76, 77, 80 and 81. The user is prompted to enter thenumber of pulses felt by the user at his fingertip. If the user entersthe correct number of pulses, the user is authenticated.

FIG. 82 illustrates two way communication scenarios to determine whethera finger exhibits properties of being attached to a live person.

FIG. 83 illustrates an example biometric sensing device, with a surfaceconfigured to receive a finger, a fingerprint sensor, and an opticalsystem.

FIG. 84 illustrates an example biometric sensing device, with a surfaceconfigured to receive a finger, ultrasound transducers, an opticalsystem and a processor.

FIG. 85 is a flowchart of a method of authenticating a user.

FIG. 86 illustrates an example interactive biometric sensing system,with a sensor, an actuator and a processor.

FIG. 87 illustrates an example interactive biometric sensing device,with a surface configured to receive an object, and a sensor that isalso configured as an actuator.

FIG. 88 is a flowchart of a method of authenticating a user.

FIG. 89 illustrates a cross sectional view of an example embodiment witha transparent ultrasound transducer array disposed between a lightsource and an optically transparent light detector.

FIG. 90 illustrates a cross sectional view of an example embodiment withan optical system including a light source and a light detector disposedbelow an optically transparent ultrasound transducer array.

FIG. 91 illustrates a cross sectional view of an example embodiment withan acoustically-transparent optical system including a light source anda light detector disposed above an ultrasound transducer array.

FIG. 92 illustrates a cross section view of an example embodiment withan optical system including a light source and a light detector disposedto a side of an ultrasound transducer array.

FIG. 93 illustrates a cross section view of an example embodiment withan optical system including a light source disposed to a side of anultrasound transducer array and utilizing a light detector disposed in aseparate device.

FIG. 94 illustrates a cross section view of an example embodiment withan optical system including a light source disposed to a side of anultrasound transducer array and utilizing a light detector disposed in aseparate device.

FIG. 95 illustrates a smart card device with a transparent ultrasoundtransducer array configured as a fingerprint scanner and with an opticalsystem disposed below the transparent transducer array.

FIG. 96 illustrates a smart card device with an ultrasound transducerarray configured as a fingerprint scanner and one or more light sourcesdisposed above the ultrasound transducer array.

FIG. 97 illustrates a mobile device with a transparent ultrasoundtransducer array configured as a fingerprint scanner and with an opticalsystem disposed below the transparent transducer array.

FIG. 98 illustrates a user device and a confirming device that may beused in a multiple device authentication process.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

The following detailed description of certain embodiments presentsvarious descriptions of specific embodiments. However, the innovationsdescribed herein can be embodied in a multitude of different ways, forexample, as defined and covered by the claims. In this description,reference is made to the drawings where like reference numerals canindicate identical or functionally similar elements. It will beunderstood that elements illustrated in the figures are not necessarilydrawn to scale. Moreover, it will be understood that certain embodimentscan include more elements than illustrated in a drawing and/or a subsetof the elements illustrated in a drawing. Further, some embodiments canincorporate any suitable combination of features from two or moredrawings. The headings provided herein are for convenience and do notnecessarily affect the scope or meaning of the claims.

This disclosure provides acoustic biometric touch scanners and methods.Ultrasound fingerprint sensing devices are disclosed. Such devices caninclude an array of ultrasound transducers configured to transmit anultrasound signal having a frequency in a range from 50 MHz to 500 MHz.The ultrasound transducers include a piezoelectric layer and a receivingsurface configured to receive a finger. The fingerprint sensing devicecan perform transmit focusing. A processor can generate an image of atleast a portion of a fingerprint of the finger based on a reflection ofthe ultrasound signal from the finger. The ultrasound transducers canalso generate a liveness parameter that can be used to authenticate thefinger. The liveness parameter can be based on a force at which a fingercontacts the surface and/or a temperature associated with the soundspeed of the reflection. In some instances, a pattern associated withthe liveness parameter can be used for authentication. Any suitableprinciples and advantages of the ultrasound fingerprint sensorsdisclosed herein can be implemented in combination with any suitablefeatures related to an integrated optical system and/or interactivebiometric sensing disclosed herein.

Ultrasonic biometric sensing devices integrated with an optical systemare described herein. The ultrasonic biometric sensing device can be atleast partially transparent such that the optical system can emit and/orreceive light through the ultrasonic biometric sensing device. Forinstance, the ultrasonic biometric sensing device can be positionedbetween the optical system and a surface configured to receive a finger.Light can be transmitted from a light source of the optical systemthrough ultrasound transducers and/or electrodes to the finger.Reflected light can propagate from the finger through ultrasoundtransducers and/or electrodes to an optical detector of the opticalsystem. The optical system can be used to generate one or more livenessparameters that can be used to authenticate the finger. In someinstances, a liveness parameter can be tracked over time and this can beused to authenticate the finger. Information generated by the opticalsystem together with the ultrasonic biometric sensing device can be usedto provide robust authentication. One or more processors can be used toauthenticate a finger based on outputs from the optical system and theultrasonic biometric sensing device.

Interactive biometric authentication is disclosed herein. Two-waycommunication can be established between an authentication device and anobject, such as a finger, being authenticated. Biometric sensing devicesdisclosed herein can detect a fingerprint and also function as anactuator that can deliver energy to the finger. Two-way communicationcan involve a real-time interactive authentication process. This canenable multi-factor authentication and provide robust authentication.Interactions with a finger for authenticating during authentication canprevent scammers or other bad actors from authenticating with priordata. In some instances, interactive biometric authentication can beperformed using an ultrasonic biometric sensing device integrated withan optical system.

Biometric Touch Scanner

Ultrasound-based fingerprint scanners can visualize not only theepidermal (superficial) layer of the fingerprint, but also the inner(dermis) layers, which makes them robust when dealing with wet hands,oil, grease, or dirt. This provides additional levels of security, andmakes them harder to spoof, which is desirable for various applications.Ultrasound-based fingerprint scanner systems can acquire 2D maps of theepidermis layers and/or 3D volumetric images of finger dermis layers.Scanning methods include impediography, acoustic microscopy, echo andDoppler imaging. The fingerprint sensing systems discussed herein canachieve a scan resolution of 500 pixels per inch (PPI) to meet FederalBureau of Investigation (FBI) and/or other standards. Such a resolutioncan translate to a lateral resolution of 50 micrometers at the focaldepth, which typically depends on the center frequency, the acousticaperture size, and the focal distance.

Other fingerprint sensing technologies can encounter challenges that maynot be present with ultrasound-based finger print scanners. Forinstance, optical fingerprint scanners can encounter challenges withresolving fingerprints with contamination. As another example,capacitive fingerprint scanners which can be forged relatively easilyvia fake fingerprint molds.

Another type of sensors is based on the concept of impediography inwhich the fingerprint surface touches the transducer elements and alterstheir acoustic impedance depending whether the surface is tissue (ridge)or air (valley). Although this technique can be convenient as it doesnot involve generating and processing ultrasound pulses and echoes, itcan be limited to acquiring the image of the fingerprint surface.Further, the impedance of a piezo-ceramic ultrasound transducer in someprevious approaches can be relatively highly sensitive to frequency. Forexample, the impedance of an element loaded by a fingerprint valley canbe approximately 800 Ohms at a frequency of 19.8 MHz and approximately80,000 Ohms at a frequency of 20.2 MHz. Similarly, the impedance of anelement loaded by a fingerprint ridge can be approximately 2,000 Ohms ata frequency of 19.8 MHz and approximately 20,000 Ohms at a frequency of20.2 MHz. This can involve multiple impedance measurements at differentfrequencies to obtain reliable measurements, which could affect theframe acquisition time. Another inconvenience with such approaches isthat the contact between the finger and the transducers can contaminateor even permanently damage the transducer surface and can affect itsperformance.

Some other approaches involve ultrasonic transducers with acousticwaveguides made from material with acoustic impedance similar to thehuman tissue to couple the ultrasound waves from the transducer array tothe finger, and using beamforming techniques to achieve the requiredresolution. Although using waveguides relaxes the frequency constraint,fabrication of waveguides typically involves additional lithographysteps, which increase the complexity and cost of the transducer design.Such approaches have achieved results that have been undesirable incertain applications. In some instances, such approaches haveencountered relatively high insertion loss that impacted the capabilityof this design even when beamforming is implemented and has increasedthe complexity of the electronics. Relatively high voltage bias andpulses, which are unsuitable for consumer electronics, have also beenused in such approaches.

The disclosed technology includes acoustic biometric touch devices thatwhen touched by naked skin scans both the outer skin layers (epidermis)and the underlying tissue (dermis and subcutis). Such sensors can beused to identify a person. The commonly used area to scan is thefingers, but any other area of the body could be scanned, for instance,soles of feet, toes, or palms. For brevity, the fingers are henceforthreferred to as the area of interest to scan. As used herein, the term“finger” encompasses a thumb. Any suitable principles and advantagesdiscussed herein can be applied to scanning any suitable area ofinterest of a human or other animal.

Biometric sensing systems discussed herein include a thin filmpiezoelectric device configured to transmit acoustic signals having afrequency in a range from 50 MHz to 500 MHz. With this frequency, animage with a desired resolution, such as 50 micrometers, can begenerated even when using an acoustic coupling layer such as glass wherethe speed of sound is relatively high (e.g., 5760 m/s). A thin filmtransducer array can be fabricated using the sputtering processes.Piezoelectric material of the transducers can be zinc oxide, aluminumnitride, or lead zirconium titanate, for example. Simulations for a ZnOtransducer device with 16 microns thickness, 0.2 mm² area, 50 Ohmssource and receiver impedance, and 1 nH inductive tuning give less than3 dB insertion loss. This can change depending on the number of elementsin the array that are used to transmit and receive. The biometricsensing device can obtain 3D ultrasound images of fingerprint layers.The biometric sensing device can implement row column addressing andbeamforming to increase the image quality and/or to reduce thecomplexity of integration and electronics design.

In combination with imaging a fingerprint, several other features arediscussed herein such as (1) finger touch force detection by measuringthe ridge widening and the fingerprint surface area, and (2) generatinga temperature of a finger and detecting the ambient temperature bymeasuring the variation in the speed of the sound which is temperaturedependent. Measuring the blood flow, heart beat/rate, and otherstructural features of biometric sensing devices are discussed herein.

The disclosed devices have the ability to scan through an intermediatemedium between the finger and the scanner. The medium could for instancebe a glass, a metal, plastic, or any suitable material that allowsultrasound propagation in a frequency range of interest. This could, forinstance, be used to make any part or the entire part of the front glasson a cell phone into a fingerprint scanner.

Since the ultrasound also enters the finger and passes through the skinlayers, a three-dimensional (3D) scan can be made of the finger surfaceand also of the internal finger tissue. Among other things, the bloodveins and/or arteries could be scanned and their blood flow could beestimated. The measurement of the blood flow can be periodic with theheart rate. Hence, the sensor can also measure the heart rate of theindividual touching the glass.

The finger print scanners discussed herein can detect one or moreliveness parameters associated with an object, such as a finger, that isin contact with a surface of the fingerprint scanner. Detecting one ormore liveness parameters can be done using an ultrasound-based sensingdevice that also generates an image of at least a portion of afingerprint. The image can have an accuracy of 500 PPI. Based on aliveness parameter, the fingerprint scanner can provide an indication ofwhether the finger is part of a live human based on the livenessparameter. This can be used to prevent molds, prosthetic fingers, orother objects from being identified as matching a fingerprint associatedwith a particular finger. The liveness parameter can be determined basedon a reflection of an acoustic wave transmitted by one or moretransducers of the fingerprint scanner. The liveness parameter can be atemperature and/or tissue stiffness, for example.

If the finger is at body temperature and the sensor is at roomtemperature, once touched, the temperature of the glass or otherintermediate layer should typically get warmer. This can generallydecrease the speed of sound in the glass. In some instances, the devicecan be warmed to a temperature greater than body temperature (e.g., bysunlight), and heat can transfer from the device to the finger to coolthe device. Hence, another objective of this disclosure is to determinethe temperature of the finger. This can be done to ascertain that it isa live finger and not a prosthetic finger that is touching the glass.When the finger is not touching the glass, the ambient temperature canbe determined by the device.

When a finger pushes harder on the glass, the contact area between thefinger and a surface, such as glass, should increase from widening ofthe ridge and/or flattening of the finger surface. This can be used todetect a force at which the finger contacts a surface. Hence, anotherobjective of this application is to measure pressure and/or force thatis applied through a calculation of the contact density with thesurface.

Pressure of the finger on the glass should increase with each heartbeatwhen the heart muscle contracts and arterial pressure increases. Betweencontractions, the heart fills with blood and pressure decreases. A timeseries of fingertip force measurements can follow a periodic, rhythmicpattern with a frequency corresponding to the person's pulse rate.Therefore, in addition to measuring temperature, the disclosed acousticscanner can estimate a pulse rate and use it to confirm that thefingertip is not a prosthetic, and is attached to a live person with ameasured pulse rate.

Further, the disclosed devices allow for tissue stiffness estimation bycomparing the above-mentioned pressure estimation with that of a directforce or pressure measurement. This can increase the certainty that theobject touching the sensor is a true finger and not a prosthetic.

The disclosed devices may also use machine learning techniques toidentify users and/or ensure the liveness of a finger or other body partbeing scanned. As an example, the devices may monitor individualcharacteristics of how a particular user interacts with and/or respondsto the device during an authentication session, in order to increase theaccuracy of the authentication. In particular, the devices may check forconsistent patterns in the user's interaction with and/or response tothe device and any stimulus provided by the response and, when suchinteraction patterns are consistent, be able to further authenticate theuser. As one particular example, the devices may monitor a user's pulseand may at least partially authenticate the user based on patterns intheir pulse (such as a resting pulse rate or other pulse attributes).

The disclosed devices can be used to identify or authenticate a personfor applications including logging in to a communications or computingdevice, logging into a software app, unlocking a door or antitheftdevice, authorizing an electronic payment, or unlocking a safety device,among other applications.

FIG. 1 illustrates an acoustic fingerprint scanning device 100, in whichan ultrasound transducer 130 (XDC) emits an ultrasound wave 135 that isreflected at the surface-finger interface 125 and also from within thefinger 105. The transducer 130 can transmit an ultrasound signal havinga frequency in a range from 50 megahertz (MHz) to 500 MHz. An acousticfingerprint scanner scans the interface between the finger 105 and themedium 120 it touches. The medium 120 can be rigid. Where the ridges 110of the finger 105 touch the surface 125, part of the acoustic wave 140will enter the finger 105 and less energy will be reflected viareflection 150. At locations where there is a valley 115 of the finger105, relatively more (for example, practically all of) the acousticenergy is reflected back to the ultrasound transducer 130 as reflection155. This contrast of reflection coefficients associated with ridges 110and valleys 115 can be used by the device to scan the finger 105surface. For instance, medical ultrasound imaging techniques can beimplemented to scan the surface of the finger 105.

Since the ultrasound wave is entering the finger 105 via the ridges 110,the scanner can also image the internal features of the finger 105. Thiscould be used for identification via pulse recognition and/or otherbiometric features such as tissue structure, ligaments, veins, andarteries. As an example, an acoustic scanning device can detect apattern of blood vessels in a finger and/or a palm of hand and identifya person based on the pattern of blood vessels. This three-dimensionalscan of the finger 105 can be used to generate two and/or threedimensional images of the finger 105. The two and/or three dimensionalimages can be processed using image processing and pattern recognitiontechniques.

In order to identify fingerprints, a finger print recognition device canresolve ridges in a finger with a resolution that is better than 50 μm.Ultrasound approaches can measure the impedance mismatch between a plate(e.g., a glass plate) and tissue, which can represent ridges, and theplate and air, which can represent valleys between ridges.

Some ultrasound scanners that use waveguides in a glass plate can guideultrasound signals with a lower frequency than the frequency of theultrasound signal from the transducer 130 (i.e., lower than 50 MHz) toallow the measurement of the impedance mismatch between a finger and theglass plate. Some ultrasound sensors can include posts of piezoelectricmaterial that are narrower than 50 μm and such scanners can measure theimpedance change due to surface contact between each post and thefinger. In ultrasound scanners with waveguides and/or narrow posts ofpiezoelectric material, relatively complex construction can be involvedto measure on a scale of 50 μm with ultrasound signals whose wavelengthis much larger than 50 μm. Such a complex assembly can result in arelatively expensive and difficult to construct system for ultrasoundfingerprint scanning. Other disadvantages of such an example can includeopacity and difficulty engineering a receiving surface incorporatingwaveguides in certain metal and/or glass surfaces . . . .

Some ultrasound scanners can generate ultrasound signals with afrequency of over 1 GHz (e.g., around 5 GHz) to avoid issues associatedwith diffraction. However, at such frequencies, the ultrasound signalsmay not penetrate tissue. Moreover, scanners with ultrasound signals atsuch frequencies can be constructed of materials that allow signals withsuch frequencies to propagate without significant attenuation.Furthermore, a two-dimensional (2D) array of 50 μm transducer elementscould be prohibitively complex for addressing, and is furthercomplicated by the electrical crosstalk while operating at frequenciesover 1 GHz.

The disclosed technology, as exemplified in the device in FIG. 2, canovercome the above-mentioned deficiencies, among others, of ultrasoundscanners that operate at lower and higher frequencies.

FIG. 2 illustrates a device 200 for focusing sound waves with arow-column addressed 2D array of ultrasound transducers. The device 200includes piezoelectric thin film 210, glass plate 220, a grating in thex-direction 230, and a grating in the y-direction 240. As shown in FIG.2, the device 200 can operate so as to have a line of focus in thex-direction 250, a line of focus in the y-direction 260, and ameasurement spot 270 where the line of focus in the x-direction 250intersects with the line of focus in the y-direction 260. Only onetransmit focal line can be active at a time. One or more receive focuslines can be active at a time. The transmit and receive focus lines areperpendicular to each other and intersect at a measurement spot with acompact focal spot size.

The device 200 uses ultrasound imaging at a sufficiently high frequency(e.g., in a range from 50 MHz to 500 MHz, such as approximately 150 MHz)in order to achieve a 50 μm resolution using beamforming. Thus, thedevice 200 can achieve a desired 50 μm resolution without waveguides.Accordingly, the device 200 can have a simpler construction relative toultrasonic scanner devices that include waveguides. Additionally, therecan be less constraint on the type of touch material and/or thickness ofthis material.

The piezoelectric layer 210 can generate an acoustic signal having afrequency in range from 50 MHz to 500 MHz. A transducer array arrangedto transmit such acoustic signals can be implemented efficiently andachieve a desired image resolution without waveguides between thetransducers and a receiving surface of the device 200, in which thereceiving surface is configured to receive and make physical contactwith the finger. In some applications, the piezoelectric layer 210 cangenerate an acoustic signal having a frequency in range from 125 MHz to250 MHz. According to certain implementations, the piezoelectric layer210 can generate an acoustic signal having a frequency in range from 50MHz to 100 MHz. For example, an acoustic signal in the range of 50 MHzto 100 MHz may be used for a device implemented in a credit card.

The piezoelectric layer 210 can include any suitable piezoelectricmaterial. For example, the piezoelectric layer 210 can be a zinc oxidelayer, an aluminum nitride layer, a lithium niobate layer, a lithiumtantalate layer, bismuth germanium oxide, lead zirconium titanate, oreven a polymer piezoelectric such as a polyvinyl difluoride layer. Thethickness of the piezoelectric layer 210 can be suitable for generatingan acoustic signal having a frequency in a range from 50 MHz to 500 MHz.The piezoelectric layer can have a thickness in a range from 3micrometers to 75 micrometers. In some applications, a zinc oxidepiezoelectric layer can have a thickness of in a range from about 10 to20 micrometers. A zinc oxide piezoelectric layer is an example of apiezoelectric layer that can be sputtered onto a substrate, such as aglass substrate. According to certain applications, a lithium niobatepiezoelectric layer can have a thickness in a range from about 5 to 10micrometers. Such a lithium niobate piezoelectric layer can be bonded toa substrate, such as a glass substrate, by an epoxy.

As illustrated, the device 200 includes a glass plate 220. The glassplate 220 is an example substrate on which a piezoelectric layer can bedisposed. Any of the acoustic sensing devices discussed herein caninclude other substrates as suitable. For example, a metal layer or aplastic substrate can be a suitable substrate in certain applications.

In an embodiment, the glass plate 220 can be about 500 μm thick. Thisthickness can be any suitable thickness based on the piezoelectricmaterial used, ultrasound frequency, and application. Such a thicknesscan be nominal for portable communication and computing device. In otherinstances, the glass plate 220 can be thinner and attached to thickerplates of any suitable material. Accordingly, the finger print devicecould be on the metal housing of a phone or any such system. Thearrangement can be any suitable size. Accordingly, the device 200 cancover most or all of a whole plate to make a touch screen and fingerprint recognition at the same time.

In operation, the device 200 can measure the reflection coefficient atthe location of the focus, that is at the measurement spot 270 at theintersection of the lines of focus 250 and 260 in x and y directions.The size of the measurement spot 270 can be determined by thediffraction resolution of the device. For instance, for ultrasonictransducers providing 150 MHz acoustic signals in glass, this spot size270 can be about 40 micrometers. The change in reflection coefficient atthe glass finger interface would be either 1 or about 0.85 depending onthe type of glass used in the finger print device. By adding a matchinglayer on the glass, between the glass and the finger, it is possible toenhance the coupling into the tissue and end up with a contrast in thereflection coefficient of approximately 1 to 0. The thickness of thematching layer can be chosen to be a quarter of a wavelength of theultrasound signal in the matching layer material. As an example, amatching layer can include epoxy with a thickness of about 5 μm for adevice 200 that transmits acoustic signals in of around 150 MHz.

The device 200 can include electronics for linear array imaging. Suchlinear imaging can be similar to linear imaging in medical ultrasoundimaging systems. In such operation, a number of elements in the arrayare grouped together and excited with different phased signals such thatthe arrival times at the plate-finger interface occur at the same timefrom all the array elements. The receiver array elements, similar innumber to the transmit array elements, would then receive the reflectedsignals, and electronic phase delays are added to each element to maketheir arrival times be the same for dynamic active focusing of tissue.Once a measurement is made at one spot, the receiving array elements canbe shifted by one element to enable a measurement of the next adjacentresolution spot. The process can be repeated to image a whole line byscanning the receive array. Next, the transmit array can be moved by oneelement, and the process can be repeated for receiving on another line.Overall, this process can be repeated to image the whole area or anydesired portion of the finger.

In the above description, the imaging is done at the plane of the platefinger interface 125. However, at the locations where the finger is intouch with the plate, ultrasound energy can penetrate into the fingerand reflections would occur from tissue inside the finger. Hence,information can be gathered from within the finger, such as informationon blood flow in capillary blood vessels in the finger. From suchinformation, the device 200 can derive information such as heart rate,indicating the subject is alive, or even some measure of capillaryvessel health based on pulse wave velocity in the vessel.

The fingerprint scanner can scan the surface and possibly the volume ofa finger. The device 200 can perform such imaging with a two-dimensional(2D) array of ultrasound transducers and no moving parts. Addressing afull 2D transducer array quickly can be a challenge due to a relativelylarge number of interconnects.

An alternative to full addressing is row-column addressing where anentire row of elements or an entire column of elements are addressed ata time. This can be achieved by having elements in the same row sharethe top (or bottom) electrode and elements in the same column share thebottom (or top) electrode. Accordingly, the transmit and receiveelectrodes can be on different sides of the array (top or bottom), andthe transmit and receive apertures can be in two different directions(row or column). This can reduce the number of interconnections that fanout from the transducer array. FIG. 3 illustrates a two-dimensional N×Nrow-column addressed array 300 of transducer elements 340 addressed by ahorizontal array of row electrodes 310 and a vertical array of columnelectrodes 320, the horizontal and vertical arrays 310 and 320,respectively, orthogonal to each other. While FIG. 3 illustrates asquare N×N array, any suitable M×N rectangular array with M rows and Ncolumns can alternatively be implemented in which N and M are differentpositive integers. FIG. 3 includes connections 330 at the ends of eachhorizontal array 310 of N×1 row arrays and each vertical array 320 of1×N column arrays.

One way of using the row-column addressed array 300 has already beendiscussed with reference to FIG. 2. One receive focus is generated pertransmit-receive event. A 512×512 2D array that uses 13 active elementsin both the transmit and receive aperture can use about 500×500=250,000transmit-receive events to make one scan. That would take approximately43 ms to complete one scan, when the coupling medium is glass with aspeed of sound of 5760 m/s and a thickness of 0.5 mm. If the receiveelectronics are expanded to record all receive elements in parallel, thescan can be completed in 500 transmit-receive events. This would abouttake about 87 μs for a full scan.

To further increase the scanning speed, parallelization can also beintroduced in the transmit stage such that multiple transmit waves areemitted at a time. To parallelize the transmit beams for row-columnaddressed arrays, the transmit beams can be separated in the frequencydomain and then received by different filters to separate them. Thedifferent filters can then pass the data to the receive beamformers. Anycombination of transmit and receive parallelization can be utilized. Theultrasound beams can also be angle steered instead of translating theactive aperture, or a combination of the two can be used. The scan canalso be synthetically focused by using synthetic transmit focusing,synthetic receive focusing, or both.

This can effectively utilize the 2D transducer array as two 1Dtransducer arrays, which are orthogonal to each other. When one array isused for transmission and the other for reception, then full 3D imagingis achievable. One interconnect and beamformer-channel can be used perrow and per column.

As an example, a fully addressed 256×256 element array can involve 65536interconnections, whereas if it is row-column addressed 512interconnections 330 can be implemented.

In operation, one set of electrodes, e.g., x-axis electrodes, can beused to transmit a line-focused beam of ultrasound. The focus can be atthe location of the surface-finger interface. Once a pulse istransmitted by the x-axis aligned electrodes, the y-axis alignedelectrodes can be used to detect a line focus in the x-direction as alsoshown schematically in the FIG. 2. The total response of the system isthe intersection of the two focal lines and results in a resolution spotcommensurate with diffraction limited resolution.

Transmit beamforming can be used for both fully addressed arrays androw-column addressed arrays to focus the emitted ultrasound beam at achosen focal depth in the medium. This maximizes the acoustic pressureat the area of interest, which improves the SNR and image quality atthat focal area. The pulse characteristics, such as length, amplitudelevel, and center frequency can also be varied to obtain the requiredimaging performance.

FIG. 4 illustrates a perspective view of a schematic of an exampleultrasound transducer array 400 on glass 410. The ultrasound transducerarray 400 includes bottom electrodes 430 on top of the glass substrate410. The bottom electrodes 430 form a vertical array of lines in ahorizontal direction. A piezoelectric thin film 420, such as zinc oxide,is on top of the bottom electrodes 430 and glass substrate 410. Thepiezoelectric thin film 420 can be square or rectangular in shape. Thepiezoelectric thin film 420 overlaps the array of bottom electrodes, butend portions of each bottom electrode 430 are not overlapped by thepiezoelectric thin film 420 as illustrated to allow for creatingmetallic contacts. The piezoelectric thin film 420 is etched withtrenches or grooves, such as v-shaped etchings 460 in both vertical andhorizontal direction to reduce crosstalk for both receiving andtransmitting in the array of transducer elements 450. The v-shapedetchings 460 are one example of trenches or grooves. Other embodimentscan include trenches or grooves that are not v-shaped and have anothersuitable shape. In certain embodiments, any other suitable transparentlayer, such as a transparent layer of plastic, can replace thetransparent layer of glass 410.

The top of each transducer element 450 is substantially square asillustrated. The top of one or more transducer elements 450 can have adifferent shape, such as a rectangular shape, in some other instances.Top electrodes 440 form a horizontal array of lines in a verticaldirection. The top electrodes 440 are orthogonal to the bottomelectrodes 430. The top electrodes 440 conform to the shape of thetransducer elements 450, including the tops of and sides of transducerelements 450 formed by v-shaped etches 460. The width of each of thebottom electrodes 430 and the top electrodes 440 substantiallycorresponds to the length of the corresponding edges of the transducerelements 450. For example, the bottom electrodes 430 each have a widthsubstantially the same as the width of the top electrodes 440, which issubstantially equal in length to each side of the square tops of eachtransducer element 450 of the two-dimensional ultrasound transducerarray 400.

The ultrasound transducer array 400 can be fabricated using apiezoelectric thin film 420 that is deposited on the surface by one ofmany thin film deposition techniques such as and not limited to:evaporation, DC or AC sputtering, sol-gel deposition, chemical vapordeposition, etc. Any suitable deposition method can provide a properlyoriented thin film that would provide a reasonable electro-mechanicalcoupling coefficient to enable the excitation and detection ofultrasound signals. In order to reach a frequency in the mid-high rangefrequencies above 50 MHz and below 500 MHz piezoelectric transducers(e.g., zinc oxide based transducers) can be fabricated using sputteringtechnology or with a relatively thin piezoelectric plate (e.g., oflithium niobate or lithium tantalate). Assuming a speed of sound of a36° rotated Y-cut lithium niobate piezoelectric layer, the plate canhave a thickness in a range from about 7.4 micrometers to 74micrometers. With a lithium niobate or lithium tantalate piezoelectriclayer, a thin bonding layer can be included between the piezoelectricand the substrate.

The frequency of operation of an ultrasonic sensor device can depend onthe material of the sensor plate or substrate to generate an image witha desired resolution. If the sensor plate is made of sapphire, the speedof sound in sapphire is relatively high at 11,100 m/sec. so a wavelength(resolution) of 50 μm involves a frequency of operation of 222 MHz. And,because sapphire has a higher mechanical impedance than mostpiezoelectric materials, the piezoelectric would operate in theso-called quarter-wavelength mode thus involving a thinner piezoelectricfilm. For instance if zinc oxide (ZnO) is being used, the thickness canbe about 7.1 μm. A similar exercise shows that for operation in quartzwith a speed of sound of 6,320 m/sec, a wavelength of 50 μm is obtainedby operating at 126.4 MHz, and a ZnO film that is about 25 μm thick. Foroperation in polypropylene with a speed of sound of 2,740 m/sec, awavelength of 50 μm is obtained by operation at 55 MHz, and ZnO filmthickness of about 57 μm. In the situations where the thickness of theparticular piezoelectric film is too large for simple deposition, otheralternative manufacturing methods such as epoxy bonding can be utilized.

The wavelength in the plate can be roughly equal to the resolution whichfor finger print recognition is 50 μm. Hence once a material is chosen,the frequency of operation is given by f=speed of sound/50 μm. Thethickness of the piezoelectric is about half a wavelength for the caseswhere the plate has a lower impedance than the piezoelectric material.Hence, the thickness of the piezoelectric is given by t=speed ofsound/twice the frequency of operation.

In an embodiment, the piezoelectric film is a ZnO thin film with athickness of about 16 μm, a transducer size of 20 μm×20 μm to 30 μm×30μm for a single sub-element, a line spacing (kerf) of 10-20 μm, a linewidth of 10-20 μm, and a pitch of 40-50 μm. The line width, linespacing, pitch, or any combination thereof can be within these rangeswhen the piezoelectric film includes a different material.

The device 400 can be made by first depositing a pattern of lines in onedirection along the glass substrate. For example, the lines and spacescan be of the order of 25 μm and can be followed by the deposition of apiezoelectric thin film about 15 μm in thickness. Another set of linesand spaces can be formed along an orthogonal direction to the previouslines and spaces between the lines. Manufacturing such a device can berelatively simple compared to other ultrasound finger print solutions.The lithography of the lines and spaces is well within the capabilitiesof current semiconductor manufacturing capabilities, and the depositionof the piezoelectric thin film (15 μm) can be done by either physicalvapor deposition (sputtering) or sol-gel manufacturing methods. Eitherway, a number of choices of piezoelectric materials are available forthis purpose. These three manufacturing steps can be used to manufacturethe device. Next, the x- and y-electrodes can be connected to electroniccircuitry for the operation of the finger print recognition.

FIG. 5 illustrates a perspective view of a portion of an ultrasoundtransducer array on a substrate. FIG. 5 illustrates a portion of theacoustic biometric touch scanner of FIG. 4, including piezoelectric thinfilm 420, top electrodes 440, transducer elements 450, and v-shapedetchings 460. FIG. 5 illustrates the v-shaped etchings 460 betweentransducer elements 450, in both horizontal and vertical directions. Thetop electrodes 440 conform to the v-shaped etchings 460 in the directionof the top electrodes 440, but do not cover or conform to the v-shapedetchings 460 in the direction orthogonal to the top electrodes 440. Asnoted above, an embodiment may include etched trenches or grooves thatare not v-shaped.

FIG. 6 illustrates an intermediate step 600 of manufacturing anultrasound transducer array, such as the scanner of FIGS. 4 and 5, bydepositing bottom electrodes on top of a glass substrate. FIG. 6includes horizontal and vertical direction views of bottom metalelectrodes 430 deposited on glass substrate 410. The bottom electrodes430 form a vertical array of horizontal elements.

FIG. 7 illustrates an intermediate step 700 of manufacturing anultrasound transducer array, such as the array of FIGS. 4 and 5, bydepositing a piezoelectric thin film 420, such as zinc oxide, over thebottom electrodes 430. The piezoelectric thin film 420 is adjacent tothe bottom electrodes 430 in areas where the bottom electrodes 430 arepresent, and is adjacent to the glass substrate 410 in areas where thebottom electrodes 430 are absent, such as between the bottom electrodes430. The piezoelectric thin film 420 may be deposited through magnetronsputtering. Parts of the bottom electrodes 430 are left uncovered at theedges on both sides to allow creation of contacts in a subsequent step.

FIG. 8 illustrates an intermediate step of manufacturing an ultrasoundtransducer array, such as the array of FIGS. 4 and 5, by etchingv-shaped etches 460 in horizontal and vertical directions on the topside of the film 420 to reduce crosstalk between elements 450. Whilev-shaped etches 460 are illustrated, any other suitably shaped etch canbe implemented. The v-shaped etches 460 form top boundaries of atwo-dimensional array of transducer elements 450 in horizontal andvertical directions. The tops of the transducer elements 450 aresubstantially square in the example of FIG. 8. The v-shaped etches 460can reduce crosstalk between different transducer elements 450 of thearray.

FIG. 9 illustrates an intermediate step of manufacturing an ultrasoundtransducer array, such as the array of FIGS. 4 and 5, by depositing topelectrodes 440 in a perpendicular direction relative to the bottomelectrodes 430. At the edges 470, the top electrodes 440 drop to thesame plane as the bottom electrodes 430 from both sides to allow makingcontacts.

In an embodiment (not shown), absorbing layers, such as rubber or epoxyloaded with particles of tungsten or silicon carbide or any suchmaterial, are placed at the edge of the plate to reduce reflections fromthe edges of the plate that may come back an interfere with the signalsof interest. Such edge reflections could also have the effect ofreducing the repetition rate at which the finger is interrogated andhence the image frame rate.

FIG. 10 illustrates an example acoustic biometric touch scanner 1000that includes an ultrasound transducer array 400 as described above withreference to FIGS. 2-9. The acoustic biometric touch scanner 100includes a receiving surface 125 to receive the touch of a person. Theillustrated ultrasound transducer array 400 is interfaced withelectronics that control its operation. These electronics includetransmit and receive circuits.

The transmit circuits excite the ultrasound transducer array 400 to emitan ultrasound beam toward the imaging area of interest. The excitationcan be created by applying an electric voltage pulse across theelectrodes of a set of transducer elements within the transducer array.The bottom electrodes can be grounded when the pulse is applied to thetop electrodes, and the top electrodes can be grounded when the pulse isapplied to the bottom electrodes. The size and shape of a transmitaperture can be varied depending on the imaging area of interest. Theillustrated transmit electronics include a transmit switching network1005, a voltage pulse generator 1010, a transmit beamforming circuit1015, and a transmit control circuit 1020.

The ultrasound transducer array includes multiple transmit channels,each associated with at least one electrode and ultrasound transducerelements. Each transmit channel can include at least a voltage pulsegenerator 1010 that generates the pulses, and a transmit control circuit1020 to provide the inter-channel phase delays when used when triggeringthe pulses. Each transmit electrode may be connected to a dedicatedtransmit channel, or multiple electrodes can be grouped together andassigned to one transmit channel. The transmit beamforming is omitted incertain implementations.

A transmit switching network 1005 can be a multiplexer that reduces thenumber of transmit channels by directing the pulses to the requiredactive elements. During row-column addressing operation, when one sideof the electrodes can operate in the transmit or receive mode, the otherside is connected to ground. This can be achieved using switches thatconnect the electrodes to ground/transmit channels, or resistors thatprovide a relatively low impedance path to connect to ground.

The transmit beamforming circuit 1015 can implement beamforming to focusthe emitted ultrasound beam at a chosen acoustic focal depth in themedium. This can produce the smallest spot size (diffraction limitedresolution) and maximum acoustic pressure at the focal line, thusproviding optimum performance at that focal area. In order to focus on aparticular focal area, beamforming circuit 1015 can include delayelements to delay channels relative to each other. For example, theultrasound transducer array 400 may transmit on multiple bottomelectrodes, for example 14 electrodes, at a time. Ultrasound transducerarray 400 may transmit over an aperture of, for example, 20 electrodes,and then shift by one or more electrodes, and transmit again. Bytransmitting over, for example, 20 electrodes at a time is a strongersignal with better signal to noise ratio than would result fromtransmitting over one electrode at a time. Beamforming circuit 1015 canadjust delays between electrodes. The voltage pulse generator 1010(pulser) can generate the pulses with a shape, length, level, frequencyand bandwidth to obtain a desired imaging performance.

As noted above, the size and shape of the transmit aperture can bevaried depending on the imaging area of interest. For example, FIG. 11illustrates a multiplexed single channel row-column addressed array 1100with an intersection 1150 of a single active row and a single activecolumn. The array 1100 includes rows 1110 and columns 1120 of transducerelements 1180, and contacts 1190. Switching network 1160 is selectingactive column 1140, and switching network 1170 is selecting active row1130. The active column 1140 and active row 1130 intersect atintersection 1150. The active aperture is the full length of lineelements. As illustrated in FIG. 11, there are two active apertures, onefor transmit and one for receive. This array can focus on a point—theintersection of the two line foci. In FIG. 11 there may be no focusingat all. Instead, the focusing can be performed as a post-processing stepof synthetic aperture focusing.

In FIG. 12, the device can include uses hardware beamformers withoutimplementing a post processing step of synthetic aperture focusing. Thearray 1105 includes rows 1115 and columns 1125 of transducer elements1185, and contacts 1195. Switching network 1165 is selecting two activecolumns 1145, and switching network 1175 is selecting three active rows1135. The active columns 1145 and active rows 1135 intersect atintersection 1155. Any suitable combination of one or more rows and oneor more columns of the transducer array can be controlled so as to beconcurrently active.

Referring back to FIG. 10, the receive circuits can process the electricradio frequency (RF) signals that are generated by the transducers inresponse to receiving the ultrasound echo signals from the medium. Thereceive circuits can then sample the signals, and digital data can beprovided to a processor 1065 that is configured to generate anultrasound image.

The illustrated receive circuits include a receive switching network1025, a low noise amplifier 1030, an analog filter 1035, a time gaincompensation circuit 1040, an analog to digital converter 1045, areceive beamforming circuit 1050, an envelope detection circuit 1055,and a receive control circuit 1060.

The receive switching network 1025 can be a multiplexer that reduces thenumber of receive channels by switching to the required receiveelectrodes. During row-column addressing operation, when one side of theelectrodes can operate in the transmit or receive mode, the other sidecan be connected to ground. This can be achieved using switches thatconnect the electrodes to ground/transmit channels, or receivechannels/ground

The received ultrasound signals can be particularly noise sensitive andat low power. The low noise amplifier 1030 can amplify the receivedultrasound signals. This first stage can influence the noise levels inthe signal, which should be sufficiently low to allow for the scan toachieve the required signal-to-noise level. The subsequent stages canvary in functionality depending on the implementation. These functionsinclude sampling and receive-beamforming. It should be noted that thesubsequent stage is the ultrasound image reconstruction processor, andthe receive circuitry can provide the processor with the sampled datarepresenting the received ultrasound echoes.

The analog filter 1035 can remove unwanted frequency components (e.g.,the analog filter 1035 can be a bandpass filter to remove unwantedfrequency components outside of a pass band). In some other instances,the analog filter 1035 can be coupled between the time gain compensationcircuit 1040 and the analog-to-digital converter 1045 and/or anadditional filter can be included between the time gain compensationcircuit 1040 and the analog-to-digital converter 1045. The time gaincompensation circuit 1040 can compensate for the increased attenuationof ultrasound signals that traveled longer distances. For example,reflections from a farther structure within a finger will attenuate morethan reflections from a surface structure of a finger. The time gaincompensation circuit 1040 can compensate by increasing the gain of thereflection from the farther structure relative to the gain of thereflection from the near structure, traveled for a shorter period oftime.

After time gain compensation, the resultant signal can be digitized byanalog to digital converter (ADC) 1045, to for subsequent digitalprocessing of the signal. In an embodiment, the analog to digitalconversion may occur at a different stage of the processing. Forexample, in an embodiment with beamforming in the analog domain, analogto digital conversion may be deferred until after receive beamforming.

The receive beamforming circuit 1050 can combine the received signalsfrom multiple receive electrodes that were amplified, filtered, andcompensated for by the low noise amplifier 103, analog filter 1035, andtime gain compensation circuit 1040, respectively. The receivebeamforming circuit 1050, which can be either an analog beamformer or adigital beamformer, can apply delays to combine the reflections receivedby the active receive electrodes, using delays for focus.

The receive control circuit 1060 can switch off the ADC 1045 or put itin standby mode so that the receive side circuits are inactive, in orderto make the ADC 1045 idle when waiting for reflections to subsidebetween consecutive measurements. This can make the acoustic biometrictouch scanner 1000 more efficient from a power consumption standpoint.

The receive control circuit 1060 can also control the timing andoperation of the receive beamforming circuit 1050. For example, receivecontrol circuit 1020 can provide the inter-channel phase delays whenrequired for received reflections of pulses by different electrodes. Inthe analog domain, beamforming could be achieved by using analog delaylines and an analog summing circuitry. The analog delay lines canprovide a desired relative phase delay between the channels, and theanalog summing circuitry will sum all analog signals to generate thebeamformed signal. That single beamformed signal can then be sampled,digitized, and sent to the processing unit for reconstruction of theultrasound image.

In some instances, the single beamformed signal can be envelope detectedin the analog domain to reduce the sampling rate requirement. Theenvelope detected, single beamformed signal can then be sampled,digitized and sent to the processor 1065 for reconstruction of theultrasound image.

Beamforming can alternatively or additionally be implemented in thedigital domain, where the individual signals of the receive channels aresampled and digitized before they are delayed and summed. The digitaldata can be relatively delayed using digital delay circuitry and summedusing a digital summing circuitry. Another approach is to acquire andstore only the samples at glass-finger interface, instead of gatheringtemporal samples along the axial direction. This can reduce thecomplexity of the hardware and could be done using similar circuitry tothe analog beamforming implementation, but instead of using a samplingcircuitry to sample all the signal at full speed, a peak detection and asingle-sample sampling circuitry can be used to detect the signal levelonly at the interface, and then use the digital data from differentactive apertures locations to generate the image of the full scan.

Envelope detection circuits 1055 can detect the envelope of thebeamformed signal to reduce the required sampling rate, and then samplethe beamformed signal.

There are multiple options for how to implement envelope detection,including peak detection using op-amps. One option is shown in FIG. 13,which illustrates peak detection circuitry 1300 using op-amps to detecta peak in an ultrasound signal. This analog-hardware peak detector canbe implemented with reduced hardware complexity and cost compared tosome other options. The peak detector is a relatively inexpensiveversion of an envelope detector and the resulting signal is thereforesimilar to the baseband signal, which can be sampled at a reducedsampling frequency.

A second option is shown in FIG. 14, in which an IQ demodulator is usedto down mix the RF signal down to base band where it can be sampled witha lower sampling frequency. FIG. 14 illustrates frequency domain plotsand signal mixing for in-phase and quadrature demodulation of a signal.Graph 1410 illustrates a signal in the frequency domain centered at 150MHz with a bandwidth of 20 MHz, sampled at 320 MHz. Block diagram 1420illustrates mixers to down mix the signal of graph 1410. Graph 1430illustrates the downmixed signal in the frequency domain, with thebaseband signal centered at 0 Hz and an image at higher (negative)frequencies. In graph 1440, the downmixed signal of 1430 is low passedfiltered, leaving the baseband signal. Graph 1450 illustrates that thebaseband signal can now be sampled at a reduced frequency.

A third option is shown in FIG. 15, which illustrates a functional blockdiagram 1500 for direct in-phase and quadrature (IQ) sub-sampling. Thisapproach can be useful if the received signal is narrow band. IQsub-sampling circuitry combines the functionality of a demodulatorfollowed by a sampling circuitry. The more narrow band the signal is,the lower the sampling rate can be while preserving the image quality.The quadrature signal is only at 90° phase shift for one singlefrequency, hence it can perform better for narrow band signals than forwide band signals.

A fourth option to reduce the data rate but still keep signalinformation is under-sampling of a band-limited signal. When sampling acontinuous time signal, its frequency content is determined by thediscrete time Fourier transform, which has images of the original signalat multiples of the sampling frequency, f_(s). FIG. 16. illustrates anoriginal high frequency signal and its spectral aliases when it isundersampled, as well as the baseband alias of the signal afterundersampling.

A fifth option is to sample the radio frequency signals at twice thefrequency of the highest frequency content in the signal. This optioninvolves a relatively high sampling rate.

Referring back to FIG. 10, the acoustic biometric touch scanner 1000includes a processor 1065 and memory 1070. The processor 1065reconstructs images from the reflected ultrasound signals that wereamplified, filtered, compensated, digitized, beamformed, sampled andenvelope detected, as discussed above. The images reconstructed byprocessor 1065 may be three dimensional images, or in two dimensions.Images may be reconstructed at different times, or in a time series, toenable change detection or video processing of the finger in two orthree dimensions.

The processor 1065 can apply image processing techniques to thereconstructed images to, for example, reduce speckle, highlight bloodvessels, measure pulse rate, estimate temperature. Memory 1070 storesreconstructed images, processing results, transmit and receive controlinstructions, beamforming parameters, and software instructions. Memory1070 can also store an image, such as a fingerprint image, that thebiometric touch scanner 1000 uses to determine if a scanned image is amatch.

Accordingly, the processor 1065 can generate an image of at least aportion of a fingerprint based on a reflection of an ultrasound signalfrom the ultrasound transducer array 400 that is reflected from a fingerat the receiving surface 125. The reflection can be received by theultrasound transducer array 400 and processed by the receive circuit.The processor 1065 can also generate additional information based on areflection of an ultrasound signal from the ultrasound transducer array400. Such additional information can include one or more livenessparameters, such a temperature associated with a finger and/or a forceat which the finger contacts the device. Based on one or more livenessparameters, the processor 1065 can provide an indication of whether thedetected image is associated with a live finger. The liveness parametertogether with the fingerprint image can be used for any suitableidentification and/or authentication applications. The processor 1065can cause this indication to be output in any suitable visual, aural, orother manner.

FIG. 17 illustrates that as the force with which a finger is pushedagainst a scanner increases, fingerprint ridges widen and the totalfinger surface in contact with the scanner increases. At lower force1700, the width of a ridge and a valley in contact with the receivingsurface is R1 and V1, respectively. At higher force, the width of theridge and valley in contact with the receiving surface is R2 and V2,respectively. The width of the ridges can be dependent on the force withwhich the finger is pushed against the scanner. At higher forces theridges widen (R2>R1) and the valleys narrow (V2<V1). For example, thewidth of the valley at lower force and the total surface in contact withthe scanner increases. In the example of FIG. 17, as force increases,the surface area of the ridge in contact with the receiving surfaceincreases. This total area, or contact density, can be measured and fromthis the applied force estimated by assuming a tissue stiffness. Thestiffer the tissue is the less it should deform under pressure.

The tissue stiffness itself can also be estimated by measuring theapplied force from the finger and comparing it with the estimated forcebased on the assumed tissue stiffness. The estimated tissue stiffness,a, is value which renders the following equation true:

F _(meas) =F _(est)(α)

where F_(meas) is the measured force and F_(est) is the estimated force.

Any of the biometric sensing devices discussed herein can implementforce detection. A biometric sensing device with force detection caninclude a processor and transducers configured to transmit an acousticsignal through a receiving surface to a finger. The processor cangenerate an image of at least a portion of a fingerprint of the fingerbased on a reflection of the acoustic signal from the finger, detect asurface area of ridges of the finger in contact with the receivingsurface based on the reflection, and estimate a force at which thefinger contacts the receiving surface based on the detected surfacearea. The processor can generate an indication of whether the finger ispart of a live human based on the estimated force. The image of at leasta portion of the fingerprint can have a resolution of 500 pixels perinch or greater.

Sound speed though a medium can change with temperature. In somematerials, the speed of sound can increase with temperature. For variousmaterials (e.g., solids such as glass, sapphire, metal, and the like),we expect the speed of sound to decrease with temperature. Accordingly,the dependence of the speed of sound in a material based on temperaturecan be used to evaluate temperature from a measurement of the speed ofsound propagating through the material.

FIG. 18 illustrates that the speed of a sound wave through a medium candecrease with an increase in temperature. The speed of sound propagatingthrough a material is typically dependent on the material temperature.The speed of sound in solids typically decreases with highertemperatures. Within the normal operating range of temperatures of abiometric sensing device, there is an approximate linear relationshipbetween the temperature and the speed of sound:

c(T)∝T×ϕ

where c(T) is a temperature dependent sound speed through a material, Tis temperature, and ϕ is a sound speed slope that is based on thematerial. The sign of ϕ can be negative or positive, but is negative invarious materials of the mediums discussed herein (e.g., glass,sapphire, and metal).

The distance between the ultrasound transducer and an acousticallyreflecting object, like the opposite glass-air interface on a smartphone, can also increases with temperature due to thermal expansion.However, the elastic constant change which results in lowering the speedof sound is typically about an order of magnitude larger than thethermal expansion coefficient and should dominate the effect ofincreasing the delay of the pulse. The speed of sound can be estimatedby the time it takes for the ultrasound signal to travel from thetransducer to the acoustically reflecting object and back again. Theshorter the time it takes, the faster the speed of sound and the warmerthe material should be. In the example of FIG. 18, spacing betweenwavefronts corresponds to wavelength. The wavelength of the reflectedultrasound beam at high temperature 1800 and low temperature 1810 is λ1and λ2, respectively, with λ2 being greater than λ1. From the speed ofsound, the material temperature can be determined based on the equationabove, analytically and/or numerically. The material temperature can bea temperature associated with the finger that can be used to generate anindication of whether the finger is part of a live human.

Any of the biometric sensing devices discussed herein can implementtemperature detection. A biometric sensing device with temperaturedetection can include a processor and transducers configured to transmitan acoustic signal through a receiving surface to a finger. Theprocessor can detect a temperature of the finger based on a sound speedassociated with the acoustic signal and generate an image of at least aportion of a fingerprint of the finger based on a reflection of theacoustic signal from the finger. The processor can generate anindication of whether the finger is part of a live human based on thedetected temperature. The processor can detect an ambient temperaturebased on a second sound speed associated with the acoustic signal whenthe finger is not in contact with the receiving surface. The processorcan detect the temperature of the finger based on a difference in soundspeed associated with the acoustic signal between when the finger is incontact with the receiving surface and when the receiving surface isuncontacted by the finger.

FIG. 19 illustrates that the time of flight from excitation until thereflected wavefront is shorter for higher temperatures. In particular,FIG. 19 shows that the time of flight from excitation until thereflected wavefront is recorded is shorter at 34° C. than at 20° C.,because the speed of a sound wave increases with an increase intemperature. Since the finger is at body temperature and the sensor atroom temperature, once touched, the temperature of the glass or otherintermediate layer would get warm and generally increase the speed ofsound in the glass. Therefore, biometric devices discussed herein candetermine the temperature of the finger based on sound speed. This canascertain that it is not a prosthetic finger that is touching the glass.When the finger is not touching the glass, the ambient temperature canbe determined by the device.

FIG. 19 compares the time of travel of ultrasound at a highertemperature of 34° C. with the travel of ultrasound at a lowertemperature of 20° C. As illustrated, the speed of sound is 4020 m/s atthe lower temperature and 4000 m/s at the higher temperature. The timeintervals are not drawn to scale to emphasize the difference in time oftravel.

At time t=0, in graphic 1910, at the lower temperature, the pulse istransmitted by the transducer towards the air/glass interface. At timet=1, in graphic 1920, the pulse nears the air/glass interface. At timet=2, in graphic 1930, a reflection of the pulse reaches the transducer.There is not activity at time t=4, in graphic 1940, since the reflectedpulse previously reached the transducer. Therefore, the time of flightfrom excitation to recording of the reflected wavefront is approximatelytwo time intervals, as illustrated in the graph 1950, at the highertemperature.

At time t=0, in graphic 1960, at the higher temperature, the pulse istransmitted by the transducer towards the air/glass interface. At timet=1, in graphic 1970, the pulse approaches the air/glass interface, butis not yet near it. At time t=2, in graphic 1970, the reflection of thepulse approaches, but has not yet reached, the transducer. At time t=3,in graphic 1970, the reflection has reached the transducer. For thehigher temperature, the time of flight from excitation to recording ofthe reflected wavefront is approximately three time intervals, asillustrated in the graph 1950. Therefore, the time of flight is shorterat the lower temperature. As time of flight varies with temperature, thetime of flight can be used to estimate relative temperatures, and oncecalibrated, can be used to estimate absolute temperature.

FIG. 20 is a flowchart of method 2000 of generating biometricinformation according to an embodiment of the disclosed technology. Inblock 2010, method 2000 transmits an ultrasound signal having afrequency in a range from 50 MHz to 500 MHz. In block 2020, method 2000generates biometric information based on a reflection of the ultrasoundsystem.

FIG. 21 is a flowchart of method 2100 of generating a biometric image.In block 2110, method 2000 transmits, using one or more ultrasoundtransducers, an ultrasound signal having a frequency in a range of 50 MZto 500 MHZ. In block 2120, method 2100 receives a reflection of theultrasound signal. In block 2130, method 2000 generates a biometricimage based on the reflection.

FIG. 22 is a flowchart of a method 2200 of manufacturing an acousticbiometric touch scanner according to an embodiment of the disclosedtechnology. In block 2210, method 2200 patterns bottom metal electrodeson top of a glass substrate in a first direction and on a bottom plane.In block 2220, method 2200 deposits a piezoelectric film over all butthe left and right edge portions of the top side of the bottom metalelectrodes. In block 2230, method 2200 etches trenches or grooves in thedeposited piezoelectric film in the first direction and a seconddirection orthogonal to the first direction. In block 2240, method 2200deposits top metal electrodes, in the second direction, conforming tothe top of the etched piezoelectric film, the top metal electrodesconforming to the left and right edge portions of the film andcontacting the bottom metal electrodes.

FIG. 23 is a flowchart of a method 2300 of detecting a temperature of afinger according to an embodiment of the disclosed technology. In block2310, method 2300 transmits, using one or more ultrasound transducers,an ultrasound signal having a frequency in a range of 50 MHz to 500 MHzthrough a medium to a finger. In block 2320, method 2300 receives areflection of the ultrasound signal. In block 2330, method 2300 detectsa temperature of the finger in response to the time of travel of thetransmitted and reflected signal.

FIG. 24 is a flowchart of a method 2400 of estimating a force at which afinger contacts a surface according to an embodiment of the disclosedtechnology. In block 2410, method 2400 transmits an acoustic signal to afinger. In block 2420, method 2400 generates an image of at least aportion of a fingerprint of the finger based on a reflection of theacoustic signal from the finger. In block 2430, method 2400 detects anarea of the finger in contact with a surface based on the reflection. Inblock 2440, method 2400 estimates a force at which the finger contactsthe surface based on the detected area of the finger in contact with thesurface.

FIG. 25 is a flowchart of a method 2500 of estimating period of a timeseries of force measurements, the period corresponding to a pulse rateestimate, according to an embodiment of the disclosed technology. Inblock 2510, method 2500 transmits a plurality of acoustic signalsthrough a receiving surface to a finger. In block 2520, method 2500generates a time series of images at a first sampling rate, each imageof at least a portion of a fingerprint of the finger based on areflection of the acoustic signal from the finger. In block 2520, method2500 detects a surface area of ridges of the finger in contact with thereceiving surface based on the reflection for each image of the timeseries of images. In block 2530, method 2500 estimates a time series offorces at which the finger contacts the receiving surface based on thedetected surface area for each image of the time series of images. Inblock 2540, method 2500 estimates a period of the time series of forces,the period corresponding to a pulse rate estimate.

FIGS. 26-34 illustrate circuits and results of simulations of envelopedetection methods for ultrasound fingerprint scanning. The simulationassumes an element width of 20 μm, a line spacing (kerf) of 20 μm, anelement height of 10 mm, and an ultrasound bandwidth of 43:8%.Excitation is a 5 cycle sinusoid at 150 MHz. Fourteen active elementsare used with an effective f #=0:893, where f # is the f-number ornumerical aperture and is defined as the focal distance divided by thediameter of the active aperture.

FIG. 26 illustrates a simulation of the one-way insertion loss.

FIG. 27 illustrates a circuit for IQ demodulation of an RF signal into Iand Q channels.

FIG. 28 illustrates an example of the simulated demodulated in-phase andquadrature signals.

FIG. 29 illustrates the response of the low pass filter used for IQdemodulation for the process of FIG. 27.

FIG. 30 illustrates an IQ demodulated envelope for a signal demodulatedby the circuit of FIG. 27.

FIG. 31 illustrates 100 MHz samples taken of the IQ demodulated envelopeof FIG. 30.

FIG. 32 illustrates a circuit for IQ sampling of an IQ demodulatedsignal.

FIG. 33 illustrates sampled in-phase and quadrature signals of an IQdemodulated signal.

FIG. 34 illustrates graphs of the envelope of an IQ demodulated signalfor IQ sampling rates of 200 MHz, 150 MHz, 100 MHz and 50 MHz.

Biometric Scanner Integrated with an Optical System

Aspects of this disclosure relate to a biometric scanner with anintegrated optical system. The biometric scanner can be at leastpartially transparent such that the optical system can emit lightthrough the biometric scanner to an object, such as a finger. Thebiometric scanner and the optical system can be used together forauthentication. The biometric scanner can be a fingerprint scanner. Theoptical system can detect a liveness parameter associated with thefinger print scanned by the fingerprint scanner. A processor canauthenticate the finger based on an image of the finger generated by thefingerprint scanner and the liveness parameter generated by the opticalsystem.

An integrated optical system can advantageously measure attributes thatare not as easily and/or reliably measured by a fingerprint scanner,such as an ultrasound finger print sensor. For instance, an opticalsystem can measure an oxygen level (e.g., SbO2) and/or respiration.Furthermore, at certain frequencies of light transmitted by an opticalsystem, measurements can be made deeper into a finger and/or illuminatedifferent attributes of a finger than various ultrasound systems.

A fingerprint scanner, such as an ultrasound fingerprint scanner, can beoptically transparent to enable multi-modality sensing using an opticalsystem. For instance, the optical system, such as a photoplethysmography device, can transmit and receive light through theoptically transparent fingerprint scanner. As an example, an ultrasoundscanner in accordance with any of the principles and advantagesdiscussed herein can include a transparent piezoelectric thin film, suchas a thin film of zinc oxide, aluminum nitride, poly-di-vinyl fluoride,or the like. In some instances, such an ultrasound sensor can includemetal electrodes that are transparent, such as indium tin oxideelectrodes or the like. In certain embodiments, a biometric sensingdevice includes an ultrasound fingerprint sensor with a flexiblepiezoelectric film and an optical system arranged to transmit and/orreceive light that propagates through the ultrasound fingerprint sensor.

A biometric scanning system that includes fingerprint scanner that is atleast partially transparent integrated with an optical system configuredto transmit and receive light through the fingerprint scanner canauthenticate a finger by detecting a fingerprint and another biologicalproperty associated with the finger to enhance strength ofauthentication. Such a system can implement a multi-modality sensor (oractuator). As an example, the optical system can generate a livenessparameter, such as a heart rate or blood oxygenation level to aid inrobustness of authentication. Multi-level authentication using afingerprint scanner and an integrated optical system provide robustapproach that can make authentication without a live finger difficult.Moreover, the integration of the fingerprint scanner and the opticaldevice in accordance with the principles and advantages discussed hereincan provide robust authentication with a relatively compact device.

A biometric scanning system that includes fingerprint scanner that is atleast partially transparent integrated with an optical system can beimplemented in any devices and/or system that use biometricauthentication, such as a finger print authentication system, a palmprint authentication system, or the like.

While example embodiments discussed below may include ultrasoundfingerprint scanners integrated with optical systems, other suitablefingerprint scanners that are at least partially transparent can beintegrated with optical systems in accordance with the principles andadvantages discussed herein. For instance, an optical system can beintegrated with a variety of fingerprint scanners, such as capacitivefingerprint scanners. Moreover, a fingerprint scanner can be transparentto any suitable energy modality that can be used to detect a livenessparameter.

FIGS. 35-45 illustrate an example embodiment of a biometric sensingdevice with an optical system 550 below an ultrasound transducer array400 with transparent electrodes 430′ and 440′. The ultrasound transducerarray 400 is below glass 410 and a receiving surface for a finger orother object to be examined. In some instances, such as in mobilephones, the glass 410 can be an engineered glass. The engineered glasscan be damage and scratch resistant. An example of engineered glass isCORNING® GORILLA® glass. While glass 410 is described with reference tothese example embodiments, any other suitable transparent material, suchas transparent plastic, can alternatively or additionally be used incertain applications. The electrodes 430′ and 440′ can be metalelectrodes used to address transducers of the ultrasound transducerarray. Other suitable metal electrodes for ultrasound transducers of theultrasound transducer array 400 can alternatively be implemented.

The optical system 550 and any other optical systems disclosed hereincan include any suitable number and/or type of light sources and/or anysuitable optical detector. For instance, the optical system 550 caninclude a reflective oximeter that includes a red LED and an infraredLED plus a photoreceptor. As another example, the optical system 550 caninclude three or more LEDs. Alternatively or additionally, the opticalsystem 550 can include one or more multispectral sensors, such as aburied quad junction photodetector. The optical system 550 can includeone or more laser light sources in certain embodiments. According tosome embodiments, the optical system 550 can include a LED and a laserlight source.

FIG. 35 illustrates an ultrasound transducer array 400 with transparenttop and bottom metal electrodes. The ultrasound transducer array 400 canbe implemented in accordance with any suitable principles and advantagesdescribed above with respect to FIGS. 2-10. The ultrasound transducerarray 400 can be transparent to light. For instance, the ultrasonictransducer array 400 can include a piezoelectric layer that istransparent, such as zinc oxide, aluminum nitride, or poly-di-vinylfluoride. The top electrodes 440 and bottom electrodes 430′ are at leastpartially transparent in the embodiment of FIGS. 35-45. Transparencyenables light to pass through the ultrasound transducer array 400components of the bottom metal electrodes 430′, the transducer material420, and the top metal electrodes 440′. The bottom metal electrodes 430′and the top metal electrodes 440′ can be implemented from any suitabletransparent metal. For instance, these metal electrodes can beimplemented by indium tin oxide.

FIG. 36 illustrates an exploded view of the transparent ultrasoundtransducer array of FIG. 35 above an optical system 550 and below glass410. Glass 410 is also transparent to light. Accordingly, lighttransmitted from the optical system 550 is transmitted through both theultrasound transducer array 400 and glass 410, which has a top receivingsurface upon which an object to be detected or scanned can be placed.Light that is reflected from the object to be detected or scanned canthen pass through the glass 410 and the ultrasound transducer array 400and to optical system 550.

Optical system 550 includes a light source 560, an optical sensor 570,and supporting electronics 580. The optical system 550 can be includedin a camera and/or a video camera in certain applications. In suchapplications, the ultrasound transducer array 400 can be disposedbetween the surface configured to receive the finger and a camera and/ora video camera. The light source 560 transmits light at one or morewavelengths or frequency bands. In some instances, the light source 560is arranged to transmit visible light. The light source 560 can transmitinfrared light in certain applications. The light source 560 cantransmit laser light in some applications. The supporting electronics580 can control the wavelength, duration, and timing of light emitted bythe light source 560. Transmitted light from light source 560 istransmitted through the transparent ultrasound transducer array 400. Anoptical sensor 570 receives light that had been transmitted by the lightsource 560. Light received by the sensor may correspond to reflectionsof light transmitted by light source 560.

Supporting electronics 580 for the optical system 550 support the lightsource 560 transmission with a light source driver, a light sourcecontrol unit, and control circuitry. The supporting electronics 580support the optical sensor 570 with a trans-impedance amplifier, asecond stage amplifier, an anti-aliasing filter, an analog to digitalconverter, and control circuitry. These supporting electronics 580components are depicted in FIG. 67, and described below in thedescription of FIG. 67,

The glass 410, ultrasound transducer array 400, and optical system 550are depicted in FIG. 36 with spatial separation so that the individualcomponents are visible. These components are integrated with each otherin a biometric sensing device.

FIG. 37 illustrates the integration of the optical system 550,ultrasound transducer array 400, and glass 410 of FIG. 36. Unlike FIG.36, the components are shown in close proximity to each other. Thecomponents are illustrated as adjoining and are not spatially separatedfrom each other.

FIG. 38 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during transmission of light at a firstwavelength λ1 from a light source 560 of the optical system 550 throughthe transparent transducer array 400 and glass 410 to a finger on thereceiving surface of the glass 410. The finger has ridges and valleysand internal structures such as veins 107. The transmitted light of thefirst wavelength λ1 may reach the receiving surface of the glass, enterfinger, and be transmitted to internal structures of the finger such asveins 107 that are relatively close to the surface of the finger. Lightmay be reflected from, for example, the internal structures of finger,such as veins 107 and from the surface of finger ridges.

FIG. 39 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during reception of reflected light at thefirst wavelength λ1 off of the finger, back through the glass 410 andthe transparent transducer array 400, to an optical sensor in theoptical system 550. The sensed reflected light can be associated withrecently transmitted light based on the wavelength of the received lightor the duration of time since light was transmitted by the light source,or any suitable combination thereof. The received light can be evaluatedbased on the amount of reflected or absorbed light in the tissue.

FIG. 40 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during transmission of light at a secondwavelength λ2 from a light source of the optical system 550 through thetransparent transducer array 400 and glass 410 to a finger on thereceiving surface of the glass 410. The transmitted light of the secondwavelength λ2 may reach the receiving surface of the glass, enterfinger, and be transmitted to internal structures of the finger such asveins 107 that are relatively close to the surface of the finger. Lightmay be reflected from, for example, the internal structures of finger,such as veins 107, and from the surface of finger ridges.

FIG. 41 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 37 during reception of reflected light at thesecond wavelength λ2 off of the finger, back through the glass 410 andthe transparent transducer array 400, to an optical sensor in theoptical system 550. The sensed reflected light can be associated withrecently transmitted light based on the wavelength of the receivedlight, the duration of time since light was transmitted by the lightsource, the distance of the path taken by the transmitted and thenreceived reflected light, or any suitable combination thereof.

FIG. 42 illustrates the light transmission and reception, respectively,at the first and second wavelengths, respectively, as illustrated inFIGS. 38-41. Comparisons in received light at different wavelengths maybe used to, for example, take a reflected pulse oximetry reading of afinger on the receiving surface. Such comparisons can be performed byany suitable processor in communication with the optical system 550.Bright red oxygenated blood absorbs more light at infrared wavelengthsthan at red wavelengths. In contrast, darker de-oxygenated blood absorbsmore light at red wavelengths than infrared wavelengths. Reflectiveoximeters can quantify this difference in absorption by measuringabsorption at red (for example, 660 nm) and infrared (for example, 940nm) wavelengths, and converting a ratio of red light measurements toinfrared light measurements to an estimate of saturation of peripheraloxygen (SpO2). Reflective oximeters can perform functions of pulseoximeters. In some instances, a pulse oximeter can be implemented inassociation with an integrated finger print sensor, such as anintegrated ultrasonic fingerprint sensor. The embodiment of FIG. 42captures measurements at two different wavelengths, such as for a pulseoximetry reading. Other applications of this embodiment may transmit andreceive light at more than two frequencies (or frequency ranges) tocharacterize differences in absorption spectra at these frequencies.

FIG. 43 is a perspective view of the example embodiment of FIG. 37 withan optical system 550 below an ultrasound transducer array 400 withtransparent metal electrodes during a transmit phase without a finger onthe receiving surface. Light emanates from a light source of the opticalsystem 550 through the transparent transducer array 400 and glass 410.There is no finger illustrated on the receiving surface of the glass410.

FIG. 44 is a perspective view of the example embodiment of FIG. 37 withan optical system 550 below an ultrasound transducer array 400 withtransparent metal electrodes during a transmit phase with a finger onthe receiving surface. Light emanates from a light source of the opticalsystem 550 through the transparent transducer array 400 and glass 410 toa finger on the receiving surface of the glass 410. The transmittedlight can reach the receiving surface of the glass 140, enter thefinger, and be transmitted to internal structures of the finger such asveins that are relatively close to the surface of the finger. Light maybe reflected from, for example, the internal structures of finger, suchas veins, and from the surface of finger ridges.

FIG. 45 is a perspective view of the example embodiment of FIG. 37 withan optical system 550 below an ultrasound transducer array 400 withtransparent metal electrodes during a receive phase with a finger on thereceiving surface. Light can be reflected off of the finger, backthrough the glass 410 and the transparent transducer array 400, to anoptical sensor in the optical system 550. The sensed reflected light canbe associated with recently transmitted light based on the wavelength ofthe received light, the duration of time since light was transmitted bythe light source, the distance of the path taken by the transmitted andthen received reflected light, or any suitable combination thereof.

FIGS. 46-55 illustrate an example embodiment with an optical systembelow an ultrasound transducer array with opaque metal electrodes. Theultrasound transducer array is below glass and a receiving surface for afinger or other object to be examined. Transparency enables light topass through the transducer material 420, even though the bottom metalelectrodes 430 and the top metal electrodes 440 are opaque.

FIG. 46 illustrates an ultrasound transducer array with opaque top andbottom metal electrodes 440 and 430, respectively. The ultrasoundtransducer array 400 can be implemented in accordance with any suitableprinciples and advantages described above with respect to FIGS. 2-10.The top metal electrodes 440 and bottom metal electrodes 430 are opaquein the embodiment of FIGS. 35-45. Transparency enables light to passthrough the transducer material 420, but light does not pass through theopaque bottom metal electrodes 430 or the top metal electrodes 440.

FIG. 47 illustrates an exploded view of the ultrasound transducer array400 of FIG. 46 above an optical system 550 and below glass 410, with theglass 410, ultrasound transducer array 400 and optical system 550. Asshown in FIGS. 48-55, the glass 410, ultrasound transducer array 400,and optical system 550 can be in close proximity to each other. Thesecomponents can adjoin and not be spatially separated from each other.Glass 410 is also transparent to light, so that light transmitted fromthe optical system 550 is transmitted through both the ultrasoundtransducer array 400 and glass 410, which has a top receiving surfaceupon which an object to be detected or scanned can be placed. Light thatis reflected from the object to be detected or scanned can then passthrough the glass 410 and the ultrasound transducer array 400, but lightdoes not pass through the opaque bottom metal electrodes 430 or the topmetal electrodes 440 to the optical system 550.

Optical system 550 includes a light source 560, an optical sensor 570,and supporting electronics 580. The light source 560 can transmit lightat one or more wavelengths or frequency band. The supporting electronics580 can cause the light source 560 to adjust one or more of wavelength,duration, or timing of transmitting light. Light transmitted from lightsource 560 is transmitted through the portions of the transducer array400 that are transparent, in the rectangular (or square) sectionsbetween opaque bottom metal electrodes 430 and top metal electrodes 440.An optical sensor 570 is arranged to receive light that had beenpreviously transmitted. Light received by the optical sensor 570 cancorrespond to reflections of light transmitted by light source 560.

Supporting electronics 580 for the optical system 550 support the lightsource 560 transmission with a light source driver, a light sourcecontrol unit, and control circuitry. The supporting electronics 580 cansupport the optical sensor 570 with a trans-impedance amplifier, asecond stage amplifier, an anti-aliasing filter, an analog to digitalconverter, and control circuitry. These supporting electronics 580components are depicted in FIG. 67, and described below in thedescription of FIG. 67,

The glass 410, ultrasound transducer array 400, and optical system 550are depicted in FIG. 47 with spatial separation so that the individualcomponents are visible.

FIG. 48 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 47 during transmission of light at a firstwavelength λ1 from a light source 560 of the optical system 550 throughthe portions of the transducer array 400 that are transparent, insections between opaque bottom metal electrodes 430 and top metalelectrodes 440, and glass 410, to a finger on the receiving surface ofthe glass 410. The transmitted light of the first wavelength λ1 canreach the receiving surface of the glass, enter the finger, and betransmitted to internal structures of the finger, such as veins 107,that are relatively close to the surface of the finger. Light may bereflected from, for example, the internal structures of finger and fromthe surface of finger ridges.

FIG. 49 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thefirst wavelength λ1 off of the finger, back through the glass 410 andthrough the portions of the transducer array 400 that are transparent toan optical sensor 570 in the optical system. The sensed reflected lightcan be associated with recently transmitted light based on thewavelength of the received light, the duration of time since light wastransmitted by the light source, the distance of the path taken by thetransmitted and then received reflected light, or any suitablecombination thereof.

FIG. 50 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 47 during transmission of light at a secondwavelength λ2 from a light source of the optical system 550 through theportions of the transducer array 400 that are transparent, in thesections between opaque bottom metal electrodes 430 and top metalelectrodes 440, and glass 410 to a finger 105 on the receiving surfaceof the glass 410. The transmitted light of the second wavelength λ2 canreach the receiving surface of the glass, enter the finger, and betransmitted to internal structures of the finger, such as veins 107 thatare relatively close to the surface of the finger. Light may bereflected from, for example, the internal structures of finger and fromthe surface of finger ridges.

FIG. 51 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 48 during reception of reflected light at thesecond wavelength λ2 off of the finger back through the glass 410 andthrough the portions of the transducer array 400 that are transparent toan optical sensor in the optical system 550. The sensed reflected lightcan be associated with recently transmitted light based on thewavelength of the received light, the duration of time since light wastransmitted by the light source, and/or the distance of the path takenby the transmitted and then received reflected light, or any suitablecombination thereof.

FIG. 52 illustrates the light transmission and reception at the firstand second wavelengths, as illustrated in FIGS. 48-51. Comparisons inreceived light at different wavelengths may be used to, for example,take a reflected pulse oximetry reading of a finger on the receivingsurface, as described above with respect to FIG. 42.

FIG. 53 is a perspective view of the example embodiment of FIG. 47 withan optical system 550 below an ultrasound transducer array 400 withopaque metal electrodes during a transmit phase without a finger on thereceiving surface. Light emanates from a light source of the opticalsystem 550 through the portions of the transducer array 400 that aretransparent and the glass 410. There is no finger on the receivingsurface of the glass 410 in FIG. 53.

FIG. 54 is a perspective view of the example embodiment of FIG. 47 withan optical system 550 below an ultrasound transducer array 400 withopaque metal electrodes during a transmit phase with a finger on thereceiving surface. Light emanates from a light source of the opticalsystem 550 through the portions of the transducer array 400 that aretransparent and the glass 410 to a finger 105 on the receiving surfaceof the glass 410. The transmitted light can reach the receiving surfaceof the glass 410, enter the finger, and be transmitted to internalstructures of the finger, such as veins 107, that are relatively closeto the surface of the finger. Light may be reflected from, for example,the internal structures of finger and from the surface of finger ridgesto an optical system below an ultrasound transducer array with opaquemetal electrodes.

FIG. 55 is a perspective view of the example embodiment of FIG. 47 withan optical system 550 below an ultrasound transducer array 400 withopaque metal electrodes during a receive phase with a finger on thereceiving surface during reception of reflected light off of the fingerback through the glass 410 and the portions of the transducer array 400that are transparent to an optical sensor of the optical system 550. Thesensed reflected light can be associated with recently transmitted lightbased on, for example, the wavelength of the received light, theduration of time since light was transmitted by the light source, andthe distance of the path taken by the transmitted and then receivedreflected light.

FIGS. 56-65 illustrate an example embodiment with an optical systemembedded within a ultrasound transducer array with opaque metalelectrodes. The ultrasound transducer array 400 with an embedded opticalsystem is below glass and a receiving surface for a finger or otherobject to be examined. The bottom metal electrodes 430 and the top metalelectrodes 440 are opaque as illustrated. In some other embodiments, thebottom metal electrodes 430 and/or the top metal electrodes 440 aretransparent or at least partially transparent. The embedded opticalsystem 550 includes embedded light sources 560 and light sensors 570.The supporting electronics 580 (not shown) can be embedded within theultrasound transducer array, to the side of the ultrasound transducerarray 400, and/or below the ultrasound transducer array 400.

FIG. 56 illustrates an ultrasound transducer array 400 with opaque topmetal electrodes 430 and bottom metal electrodes 440 with embedded lightsources 560 and light sensors 570. The ultrasound transducer array 400can be implemented in accordance with any suitable principles andadvantages described above with respect to FIGS. 2-10. Light sources 560and sensors 570 are embedded within the ultrasound transducer array, forexample, within the rectangular (or square) sections between opaquebottom metal electrodes 430 and top metal electrodes 440. Light from thelight source 560 and light to the sensors 570 no longer needs to passthrough the entire ultrasound transducer array 400, since the opticalsystem 550 including the light sources 560 and sensors 570 are embeddedwithin the ultrasound transducer array 400. In an embodiment where thelight source 560 and/or the optical sensor 570 is at the top surface ofthe transducer material 420 and between the top metal electrodes 440,light need not travel through the transducer material 420, bottom metalelectrodes 430, or top metal electrodes 440.

FIG. 57 illustrates an exploded view of the ultrasound transducer array400 with embedded light sources 560 and light sensors 570 and the glass410 of FIG. 56. As shown in FIGS. 58-66, the glass 410 and ultrasoundtransducer array 400 with embedded light sources 560 and sensors 570 areintegrated with each other.

FIG. 58 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during transmission of light at a firstwavelength λ1 from a light source 560 through the glass 410 to a fingeron the receiving surface of the glass 410. The transmitted light of thefirst wavelength λ1 can reach the receiving surface of the glass 410,enter the finger, and be transmitted to internal structures of thefinger, such as veins 107, that are relatively close to the surface ofthe finger. Light may be reflected from, for example, the internalstructures of finger and from the surface of finger ridges to theoptical sensor 570.

FIG. 59 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during reception of reflected light at thefirst wavelength λ1 off of the finger, back through the glass 410 to anoptical sensor 570 in the optical system. The sensed reflected light canbe associated with recently transmitted light based on the wavelength ofthe received light, the duration of time since light was transmitted bythe light source, and the distance of the path taken by the transmittedand then received reflected light, for example.

FIG. 60 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during transmission of light at a secondwavelength λ2 from a light source 560 through the glass 410 to a fingeron the receiving surface of the glass 410.

FIG. 61 illustrates a cross sectional view of the integrated optical andultrasound system of FIG. 57 during reception of reflected light at thesecond wavelength λ2 off of the finger, back through the glass 410 to anoptical sensor 570 in the optical system. The sensed reflected light canbe associated with recently transmitted light based on the wavelength ofthe received light, the duration of time since light was transmitted bythe light source, and the distance of the path taken by the transmittedand then received reflected light, for example.

FIG. 62 illustrates the light transmission and reception at the firstand second wavelengths, as illustrated in FIGS. 58-61. Comparisons inreceived light at different wavelengths may be used to, for example,take a reflected pulse oximetry reading of a finger on the receivingsurface, as described above with respect to FIG. 42.

FIG. 63 is a perspective view of the example embodiment of FIG. 57 withan optical system embedded within the ultrasound transducer array 400with opaque metal electrodes during a transmit phase without a finger onthe receiving surface. Light emanates from light sources 560 of theoptical system 550 through the glass 410. There is no finger on thereceiving surface 125 of the glass 410.

FIG. 64 is a perspective view of the example embodiment of FIG. 57 withan optical system embedded within the ultrasound transducer array 400with opaque metal electrodes during a transmit phase with a finger onthe receiving surface. Light emanates from light sources 560 of theoptical system 550 through the glass 410.

FIG. 65 is a perspective view of the example embodiment of FIG. 47 withan optical system 550 embedded within the ultrasound transducer array400 with opaque metal electrodes during a receive phase with a finger onthe receiving surface.

FIG. 66 illustrates a cross sectional view of an example embodiment withlight sources 560 and light sensors 570 lateral to and adjoining anultrasound transducer array 400. In the embodiment of FIG. 66, the lightsources 560 can emit light in a direction substantially parallel to thereceiving surface. Light emitted by the light sources 560 is transmittedthrough the transparent transducer material 420 to grooves 460 in theultrasound transducer array 400, through glass 410 to the receivingsurface 125 of glass 410 with a finger. The grooves 460 can turn thelight emitted by the light sources 560 toward the receiving surface. Thetransmitted light can reach the receiving surface of the glass 410,enter finger 105, and be transmitted to internal structures of thefinger, such as veins 107, that are relatively close to the surface ofthe finger. Light may be reflected from, for example, the internalstructures of finger and from the surface of finger ridges. Suchreflected light may pass through the glass 410 to grooves 460 betweenthe upper metal electrodes and to the light sensors 570 lateral to andadjoining the ultrasound transducer array 400. The grooves 460 can turnthe reflected light toward the optical sensors 570.

In addition to the light sources 560 and light sensors 570 lateral toand adjoining the ultrasound transducer array 400, the optical system550 includes supporting electronics 580 (not shown), which may be to theside of the ultrasound transducer array 400 with the light sources 560and light sensors 570 and/or below the ultrasound transducer array 400.

FIG. 67 illustrates an example acoustic biometric touch scanner,including an ultrasound system and an optical system 550. The ultrasoundsystem includes an ultrasound transducer array 400, transmitelectronics, and receive electronics. The optical system 550 includes alight source 560, an optical sensor 570, and supporting electronics 580.

The illustrated transmit electronics include a transmit switchingnetwork 1005, a voltage pulse generator 1010, a transmit beamformingcircuit 1015, a transmit control circuit 1020. The illustrated receiveelectronics include a receive switching network 1025, a low noiseamplifier 1030, an analog filter 1035, a time gain compensation circuit1040, analog to digital converter 1045, a receive beamforming circuit1050, an envelope detection circuit 1055, a receive control circuit1060, a processor 1065, and a memory 107. The receive electronics and/orthe transmit electronics can be implemented in accordance with anysuitable principles and advantages described above with respect to FIG.10.

The illustrated optical system 550 includes a light source 560, anoptical sensor 570, and supporting electronics 580. The illustratedsupporting electronics 580 supports the light source 560 with a lightsource driver 581, a light source current control unit 582, and controlcircuitry 583. The illustrated supporting electronics 580 supports theoptical sensor 570 with control circuitry 583, a trans-impedanceamplifier 584, a second stage amplifier 585, an anti-aliasing filter586, and an analog to digital converter 587. In an embodiment, theoptical system 550 corresponds to, and has corresponding components to,a reflectance pulse oximeter.

Control circuitry 583 controls the timing, duration, wavelengths(wavelength ranges), and power of emissions of light from the lightsources in the illustrated optical system 550. The control circuitry 583can also control timing of sensing incident light that may bereflections of previously transmitted light. The control circuitry 583is shown as a single block in FIG. 67. However, the control circuitry583 can function to control transmission and reception may be divided,or may be combined with the processor 1065, receive control circuit1060, and or transmit control circuit 1020 of the ultrasound system. Inan embodiment, the control circuitry 583 coordinates with the processor1065, transmit control circuit 1020 and receive control circuit 1060 tocoordinate and control the relative timing of ultrasound transducerarray 400 transmissions and light source 560 light emissions. Forexample, in an embodiment, the ultrasound and light source emissions maybe controlled to not overlap. In another embodiment, the ultrasound andlight source emissions may be controlled to at least partially overlap.

The light source 560 may include any suitable light source. Examplelight sources include light emitting diodes, organic light emittingdiodes, lasers, and the like. For instance, the light source 560 caninclude one or more light emitting diodes configured to emit light overa range of frequencies, and of durations and power levels to obtainbiometric measurements. Example biometric measurements include a pulseoximetry reading, blood flow measurements, a pulse reading, temperature,glucose detection, blood glucose level, dehydration level, blood alcohollevel, and blood pressure. For example, LEDs that emanate light in thevisible and infrared portions of the spectrum may be used to obtainbiometric measurements. The light source 560 can emit a variety ofdifferent wavelengths in certain applications and is not limited tospecific wavelengths in such applications. For example a mid-IR laserpulse can be used for glucose detection.

The light source 560 may include multiple light sources arranged in aline, in rows and columns, in a hexagonal tessellation, or othersuitable arrangements. An individual light source 560 or multiple lightsources may be included in optical system 550. In an embodiment, asingle light source 560 may be below the ultrasound transducer array400, as shown in FIGS. 37 and 47. In an embodiment, individual lightsources may be embedded in the ultrasound transducer array 400, in theportions between the top metal electrodes 430 and bottom metalelectrodes 440, for example, as shown in FIG. 56. In an embodiment,light sources may be to the sides of the ultrasound transducer array400, for example, as shown in FIG. 66. In an embodiment, opticalfiber(s) or other suitable light turning features can guide lightinward.

The light source current control unit 582 controls start time, end time,wavelength and power of pulses of light to be emitted by the lightsource 560. The light source driver 581 sends the pulses to the lightsource 560.

Emitted pulses are transmitted by the light source 560 that may bereflected back to the optical sensor 570.

The optical sensor 570 can include any suitable optical sensingelements. For instance, the optical sensor 570 can include one or morephotodiodes configured to receive light over a range of frequencies, andof durations and power levels to obtain biometric measurements, such asa pulse oximetry reading and/or blood flow measurements. For example,photodiodes that receive light in the visible and infrared portions ofthe spectrum may be used to obtain biometric measurements. The opticalsensor 570 can sense a variety of different wavelengths in certainapplications and is not limited to specific wavelengths in suchapplications. In an embodiment, the optical sensor 570 can include amultispectral imager capable of sensing light at multiple frequencybands. In an embodiment, the optical sensor 570 can be included in avideo camera capable of measuring subtle changes in color in the skinstemming from the flow of blood to detect a pulse.

The optical sensor 570 may include multiple optical sensors arranged ina line, in rows and columns, in a hexagonal tessellation, or othersuitable arrangements. An individual optical sensor 570 or multipleoptical sensors may be included in optical system 550. In an embodiment,a single optical sensor 570 may be below the ultrasound transducer array400, for example, as shown in FIGS. 37 and 47. In an embodiment,individual optical sensors 570 may be embedded in the ultrasoundtransducer array 400, in portions between the top metal electrodes 430and bottom metal electrodes 440, for example, as shown in FIG. 56. In anembodiment, optical sensors 570 may be to the sides of the ultrasoundtransducer array 400, for example, as shown in FIG. 66.

In certain embodiments, the optical sensor 570 includes one or morephotodiodes that that convert incident photons to a current. Theresultant current is converted to a voltage by trans-impedance amplifier584, and amplified by a second stage amplifier 585. An anti-alias filterlow 586 pass filters the amplified voltage from the second stageamplifier to reduce aliasing. The output of the anti-aliasing filter isdigitized with an analog to digital converter 587. The analog to digitalconverter 587 for the optical system 550 can operate at a lowerfrequency than the ultrasound system analog to digital converter 1045.For example, the optical system 550 analog to digital converter 1045 canoperate in, for example, the 2 kHz range, with approximately 22 bitdigitization accuracy. Other suitable frequency ranges and/or accuraciescan be implemented for particular applications. The analog to digitalconverter 587 can digitize the analog output of the anti-aliasing filter586 for subsequent digital processing of the signal. In an embodiment,the analog to digital conversion may occur at a different stage of thereceive processing chain

FIG. 68 illustrates an example biometric touch scanner, including anultrasound system and an optical system. The ultrasound system includesan ultrasound transducer array, transmit electronics, and receiveelectronics. The optical system includes a light source, an opticalsensor, and supporting electronics. The components of FIG. 68 can beimplemented as described above with respect to FIG. 10 and FIG. 67. Thebiometric scanning device of FIG. 67 includes separate components forthe ultrasound system and the optical system. In contrast, the biometricscanning device of FIG. 68 includes a shared processing unit for theultrasound system and optical system. The processor 1065 of FIG. 68 canserve as a processor for the ultrasound system and control circuitry forthe optical system.

One aspect of the disclosed technology is a biometric fingerprintsensing device. The device includes an optical emitter configured totransmit light having a frequency in a range from 400 nm to 1000 nm. Thedevice further includes an array of ultrasonic transducers configured totransmit an ultrasound signal having a frequency in a range from 50megahertz (MHz) to 500 MHz. The ultrasonic transducers include apiezoelectric film. The device further includes first metal electrodes.The device further includes second metal electrodes orthogonal to thefirst metal electrodes. The first metal electrodes and the second metalelectrodes enable addressing of the ultrasonic transducers of the array.The device further includes a surface configured to receive a finger.The device further includes a processor configured to generate an imageof at least a portion of a fingerprint of the finger based on areflection of the visible light and/or the ultrasound signal from thefinger. The device further includes an actuator configured to vary thetemperature of and/or the pressure on the finger in contact with thereceiving surface.

In an embodiment, the optical emitter transmits light through theultrasonic transducer, the first metal electrodes, and the second metalelectrodes. In an embodiment, the ultrasonic transducer is at leastpartially transparent In an embodiment, the first and second metalelectrodes are at least partially transparent.

In an embodiment, the array of ultrasonic transducers are between theoptical emitter and the receiving surface, and the ultrasonictransducers, first metal electrodes, and second metal electrodes are atleast partially transparent to the transmitted light.

In an embodiment, the ultrasound transducers transmit the ultrasoundsignal through the optical emitter.

In an embodiment, the optical emitter comprises an array of opticalemitters in squares on a plane, each square bounded by projections ofthe first metal electrodes and second metal electrodes to the plane.

In an embodiment, the optical emitter adjoins the ultrasound transducer.

Interactive Biometric Scanner

Embodiments of this disclosure relate to a scanner, such as a fingerprint scanner, with the ability to become an actuator. The actuator candeliver energy to an object, such as a finger. This can establish atwo-way communication between the scanner and a user. Such two-waycommunication can involve real-time interactive authentication process.Authentication with two-way communication can be performed on atimescale of milliseconds for any suitable interactive biometric scannerdisclosed herein. For instance, any suitable ultrasonic interactivebiometric scanner with ultrasonic sensing and/or actuation disclosedherein can perform two-way authentication on a millisecond timescale.Some other two-way communication techniques disclosed herein can beperformed in several seconds. Two-way communication can provide robustauthentication. The two-way communication can be referred to asinteractivity.

Two-way communication can enable multi-factor authentication thatprovides real-time interaction aimed at defeating a scammer who mighthave secretly and/or illegally copied or otherwise obtained data thatrepresents a user's fingerprint scan and/or other biological information(e.g., pulse and temperature from a previous authentication session)from authenticating. Interactions with a finger or other object forauthenticating during authentication that are not predictable in advance(e.g., that are random) should be able to prevent scammers fromauthenticating with prior data. As one example, while having afingerprint and pulse scanned, a user being authenticated could beprompted to stand while keeping a finger on the scanner. The scannercould then detect a change in pulse associated with standing and thechange in pulse could be used to authenticate the user. As anotherexample, haptic energy could be felt by a user and could prompt the userto take action, such as removing a finger from the scanner or changing aforce applied to by the finger to the scanner. Such action could bedetected by a sensor, such as an ultrasound sensor, by a variety ofmethods, such as by detecting widening of the ridges (e.g., forcedetection) and/or capillaries of the finger.

Fingerprint scanners, such as an ultrasound fingerprint scanner, canimplement a sensor and also an actuator. As an actuator, the scanner candeliver energy, such as ultrasound in a form that can be felt by theuser. This can establish a two-way communication between the scanner andthe user. The scanner can detect a response to the delivered energy foruse in authentication. As an example, ultrasound energy can be deliveredthrough a focus burst that can be detected by the finger as a pulse ofheat, a push, a neuro-modulation such as a tingling sensation in thefinger, the like, or any suitable combination thereof. Other energymodalities can alternatively or additionally be used to deliver a signalthat can be sensed by the finger. Accordingly, the idea of converting ascanner into a two-way communication device can apply to a variety offorms of scanners. In some instances where the sensor also functions asan actuator, the sensor can detect the response to the delivered energy.For instance, an ultrasound fingerprint sensor can also detect how avariety of liveness parameters (e.g., tissue stiffness and/ortemperature) change in response to energy delivered by the ultrasoundfingerprint sensor.

The sensor and actuator can be implemented below a surface configured toreceive a finger for authentication. The sensor and actuator can bepositioned below one or more layers of glass and/or engineered glass.Interactive biometric authentication can be performed while a finger ispositioned on a surface of a device that is authenticating the finger.

Certain actuators can deliver energy that is perceptible to a personbeing authenticated. As one example, an ultrasound fingerprint sensorcan provide haptic energy to a finger that a person can perceive toprompt a response from the person (e.g., standing up). Some otheractuators can delivery energy that is imperceptible to a person beingidentified. For instance, an ultrasound fingerprint sensor can generateheat to a degree that a person does not perceive but that can bedetected as a response used to authenticate the finger.

Two-way communication can be implemented by one integrated device withdifferent aliveness measurements from the same location as an ultrasoundsensor where the finger is being scanned.

Two-way communication can be implemented using different alivenessmeasurements and interactivity can be performed at the same time that afingerprint and at the same location. This can ensure that the differentaliveness parameters are associated with the same user.

Two way communication can be implemented using different devices toauthenticate a person. For example, a person being authenticated can beprompted by one device to interact with a different device. In anembodiment where the two devices are a phone and a tablet, a person canbe prompted by the phone to press his finger on the tablet.

With two-way communication, safety and/or security of the scanner can beenhanced. With an interactive authentication session or process, a fakeauthentication with a scammer using a past sign-in can be prevented. Asignal sent by the scanner during authentication can result in the userresponding with further action for multi-factor authentication. Thesignal can be sent at a time that is not easily predictable in advance.The actuator can prompt a variety of responses. Some responses can beinvoluntary. For instance, a finger can be heated in response to energybeing applied to the actuator. Certain responses can involve voluntaryuser action. For example, the response can be to have another fingerscanning as agreed upon by prior agreement, entering a set of numbers,or any other possible actions or many actions all agreed upon earlierwhen the system was being set up. Other types of excitation could becreating Braille characters under the finger that can be a codeeliciting a coded response form the user. These authentication promptscan be generated at random without prior agreement and can be recognizedand recorded by a processor, memory, and associated software. Such aprocessor, memory, or associated software can also embody one or moresuitable features of the technology disclosed herein. Detecting any ofthese types of responses can create a more secure method ofauthentication relative to just detecting a fingerprint.

Fingerprints do not typically change. Accordingly, if a fingerprint isspoofed, it cannot be easily changed like a password. With two-waycommunication, aspects of a live finger or live user can be used as anadvantage in providing robust authentication. Certain actuation cancreate involuntary biological responses that can be detected and used inmulti-factor authentication that can be difficult to spoof. In someinstances, responses to actuation can engage a user's brain such thatthe user performs a certain action that can be detected and used inmulti-factor authentication. Such responses can be particularlychallenging to spoof.

The principles and advantages of the two-way communication discussedherein can be implemented in any suitable device, system, or methodwhere authentication is used to access secure data or information.Example applications include secure authentication for electronicdevices, guns, and keys for doors in cars or homes. Some otherapplications include smart cards (e.g., a credit card that includes anintegrated chip). Any suitable principles and advantages discussedherein can be implemented in a smart card. A smart card can include achip that includes a fingerprint scanner, such as an ultrasonicfingerprint sensor. In some instances, a smart card can include apiezoelectric film (e.g., a zinc oxide film) over most or all of a majorsurface of the card for ultrasonic biometric sensing. Some additionalapplications include the steering wheel of a car that checks a driver'sbiometric information (such as heart rate) to determine whether thedriver appears to be angry, anxious, intoxicated, the like, or anysuitable combination thereof.

Therefore, the “aliveness” measurements enabled by the disclosedtechnology allow for interactivity. This allows real-time interactionsfor robust authentication aimed at defeating attempts to gainunauthorized access by a scammer who obtained a digital representationof, for example a user's fingerprint scan or, retinal scan, pulse,temperature, electrocardiogram or other identifiers for a user. Suchsecurity systems can be used to control access to a web site database,building, or use of a weapon, for example. If random interactions arerequired during the authentication process, a copy of an earlierauthentication session would not be sufficient to gain access.

The disclosed technology includes systems that can use one more of thefollowing three types of authentication factors: knowledge factors,possession factors, and inherence factors. Knowledge factors includepasswords and personal identification numbers (PIN) that are (or shouldbe) known only to an individual user. Possession factors include keys,fobs, smartphones, or a physical cryptographic key like YubiKey.Inherence factors or biometrics include fingerprint, iris, or facialscans, gait, heartbeats, or other biometric indicators. Systems thatrely on just one of these three types of factors may be more vulnerablethan systems that use two or more of these factors. For example,passwords can be stolen, fobs can be cloned, and biometrics imitateddigitally.

Two-factor authentication can involve authentication from two of thethree types of authentication factors. For example, a two factorauthentication system may involve authentication based on a userentering a knowledge factor, such as a password, and having a possessionfactor, such as a cell phone, Yubikey, or Duo app. In this example, anattacker with a user's password can be defeated if he does not have theuser's cell phone or fob. Similarly, an attacker with the user's cellphone, but without his password, would not be able to authenticate.

Three-factor authentication can involve authentication from all threefactors: knowledge, possession, and inherence. Surface-only fingerprintshave been used for decades to identify crime suspects. Such fingerprintscan be lifted from surfaces, as depicted in Mission Impossible and JamesBond movies, or digitally copied to fool authentication systems. Threedimensional fingerprints that capture internal structural features aswell as fingerprint surfaces are not as easily lifted from surfaces ordigitally copied. An extra level of security is introduced by usinginherence or biometric features that change in response to stimuli, asan attacker would not necessarily know in advance which stimuli will beapplied during the authentication process.

One concern with introducing fingerprint or iris scan authentication isthat an attacker in, for example, a war zone might remove a body part inorder to defeat an authentication system. This grisly possibilityunderscores the importance of testing whether the scanned finger isintact and attached to a live user, as opposed to an amputated finger, afinger of a dead person, a prosthetic finger, or a digitalrepresentation of a finger.

The disclosed technology includes methods to measure liveness withactuators that activate neurons via radiation, pressure or heat, causinga change in an inherent feature or biometric. The disclosed technologyfurther includes methods to measure liveness by prompting a user to takean action. This interaction with the user's brain help determine thatthe finger is a live finger attached to a user, as there is aneurological connection to the brain of the user, to help verify theinherence factor.

For example, a user being scanned by an embodiment of the disclosedtechnology can have her pulse scanned while the fingerprint is beingauthenticated, and for two factor authentication the user can be askedto stand while holding on to the scanner. The device would measure achange in heartbeat and/or another parameter that would change a resultof standing. Haptic energy could be felt by a user and be a prompt forthe user to take an action, such as removing her finger, pushing down onthe scanner, which would be registered as a force applied by the fingerto the scanner, or typing a letter or word on a screen. The hapticenergy can be actuated by a MEMS device and controlled by a controller,such as an ASIC. Involuntary or voluntary finger movement in response tothe haptic energy can be detected by a fingerprint sensor.

In certain applications, a user being scanned by an embodiment of thedisclosed technology can have her blood oxygen level taken after beingprompted to take several deep breaths, which should increase her bloodoxygen level. Alternatively, she can be prompted to hold her breath,which should reduce her blood oxygen level. A change in blood oxygenlevel can be determined based on a comparison of blood oxygen readingsbefore and after a prompt. In an embodiment, the breathing patterns of auser being scanned can be correlated with heart rate, such as aninstantaneous heart rate. In an embodiment, changes in the breathingpatterns of a user being scanned are analyzed in response to prompts,and correlated with instantaneous heart rates before and after theprompt.

Digital representations of a previous session of an “alive” andauthorized user, including her fingerprint, retinal scan, pulse or othermeasurements, could be used to gain access in some instances inpredictable authentication sessions. By randomly or otherwiseunpredictably varying, for example, the haptic energy pattern and timingfrom session to session, the digital representations of prior sessionsshould not be sufficient to gain unauthorized access.

Some users are reluctant to use their fingerprint to unlock a portablecomputing device, such as a smart phone, since unlike a password, afingerprint is not easily changeable. Stolen passwords can easily bereplaced, but once someone has stolen the digital likeness of theirfingerprint, the fingerprint is not easily changed. The disclosedtechnology incorporates the uniqueness of a fingerprint from a negative(for those who worry their ‘likeness’ could be stolen), into a positivewith ways to mitigate the issue of someone copying the fingerprint byadding ‘aliveness’ measurements as well as a random or unpredictableauthentication operation that involve a user's unique fingerprint orother characteristics of the finger.

One solution to authenticating that the user is the user, and not ascammer sending the bank the 0s and 1s that represent the user'spassword or an advanced biometric representation, is to add secondfactor authentication (2FA) to verify the user. Some current 2FA usesboth a password and a second factor for authentication. Examples of asecond factor include, for example, a number texted to the user's mobilephone, a code emailed to the user, or correct answers to questions thatwere previously provided by the user. Some second factor authorizationsmake use of a second device.

However, existing 2FA methods can be hacked, including random numberstexted to a second device that are then entered by the user, afterinputting their password. Furthermore, existing 2FA or multi-factorauthentication methods can take time or be difficult for a user, orrequire a second device, resulting in low usage by users of multi-factorauthentication methods.

The disclosed technology includes interactive approaches in which thedevice can affect a fingerprint, akin to a bright light causing aninvoluntary muscle contraction in the pupil that is being scanned or afacial recognition system recognizing a closing of an eye following anauthentication command to close an eye. The disclosed technologyincludes an actuator that interacts with the user, causing a detectableresponse. The interaction can include heating, cooling, vibrating,shining a light, emitting a sound, or any other suitable stimulus thatimpacts a user. The actuator can be implemented by a fingerprint sensor.In some instances, the actuator can include hardware of a computingdevice, such as a mobile phone, that is separate from the fingerprintsensor and performs other functionality for the computing device (e.g.,vibrating the device). The response can be involuntary or voluntary.Involuntary responses, such as a change in heartbeat or pulse rate maynot even be apparent to the user. Other involuntary responses may relateto changes in fingerprint ridges or internal fingerprint structures.Voluntary responses include a directed finger movement, finger pressure,or entering information into a user interface by touch, typing orspeech.

As the stimuli can be unpredictable (e.g., random and/or exhibitstatistical randomness), can be in multiple steps, and can elicitpredictable voluntary or involuntary responses, the disclosed technologyenables robust authentication. The stimuli or method sequence can berandomly chosen by, for example, a randomized algorithm and/or using arandom number generator. For example, haptic stimuli can beunpredictably varied with respect to interval between vibrations, thenumber of vibrations, and the duration of vibrations, location ofstimuli, or pattern of stimuli. Similarly, different heating stimuli arepossible. The user can also be prompted to rapidly rub his finger on thepalm of his other hand and then put it back on the fingerprint sensor.The higher temperature could then be measured. This type of interactionwould be very difficult to predict. The user can be prompted to reorientthe device, move in a particular direction, or squeeze a finger. Each ofthese actions result in predictable outcomes that can be sensed andanalyzed, by, for example, quantifying finger ridge spacing or movementof a point on the finger from frame to frame of an acquired set ofultrasound images of the finger.

In an embodiment, a user can be authenticated using two collocateddevices, such as a computer in a bank branch and the user's mobilephone, both of which include biometric touch scanners in accordance withany of the principles and advantages disclosed herein. The user is thenauthenticated by confirming that the devices are reading fingerprints ofthe same user with corresponding measure(s), such as temperature of theuser, pulse rate, and/or pulse oximetry reading, to authenticate theuser.

This approach can be expanded to two users in the same place, in asystem analogous to a bank requiring co-signing wires above a certaindollar threshold. Both parties can sign on using an application,indicating one is the account holder and the other a co-signer. Bothparties get their fingerprint verified as being their own as well as oneor more of their biometric parameters, such as one or more of theirpulse, temperature, and oxygen level. The parties switch devices whenprompted by the bank or other online site, and fingerprint and one ormore biometric parameters are detected for each user after switching.The users are authenticated if the fingerprint and biometric levelsmatch.

An added level of security may be possible by introducing a digitalwatermark and encryption, in which the biometric data and interactionsare combined to form a digital watermark and encrypted within the datastream. For example, such a digital watermark and encryption can beintroduced in the data streams of the systems of FIG. 67 and/or FIG. 68.A watermark and encryption can enable the detection of a cut-and-pasteaction by helping one notice a disruption or pause or break in the datastream. For example, if a bank causes a haptic energy pulse prompt for auser to recognize that they should remove their finger, then the scammerwould have to almost instantly send the 0s and 1s equivalent to removinga finger and then also send the requisite 0s and 1s in response to aprompt of standing that would change the pulse—or the 0s and 1srepresenting the user's fingerprint being pushed down—to beauthenticated. A watermark and encryption would be designed to be ableto detect splices or interruptions in the flow of data, as the scammertried to keep up with interactive prompts.

Embodiments of the disclosed technology related to improved devices,systems, and methods for authentication, including authentication todetermine aliveness. Such embodiments relate to determining whether afingertip on a receiving surface of a sensing device is an actual livefinger instead of a prosthetic device, non-live finger, or other objectthat is attempting to authenticate in place of a live finger.

The disclosed technology can address problems related to scammersspoofing remote devices with so-called ‘replay’ attacks. For example ascammer may try to trick an authentication method by replaying thedigital signature used during authentication sessions from the newerbiometrics scans. Moreover, certain biometric scans may not stopscammers using a Trojan horse to add a false template (i.e., a templatethat would be matched with a fingerprint or face algorithm uponauthentication) to circumvent the authentication process. With two-waycommunication between a scanner and a user, it can be significantly moredifficult to trick a scanner with a digital signature or a falsetemplate or other attempt to trick the scanner.

During transmission a scammer could also attack the transmission betweenthe phone and the bank and tamper with the template stored at a bank orcloud site. Computing devices that include bio-metric scans in additionto a fingerprint scan, such as facial recognition or iris scanning,still can be fooled with false templates or replay attacks.

Therefore, there is a need for simple-to-execute systems ofinteractivity, with multiple scans happening during an authenticationsession as a way to overcome these authentication problems. Thedisclosed technology combines biometric measurements and factors withinteractions between different biometric devices for authentication.

Just as Mission Impossible agents and James Bond replicated fingerprintsto fool authenticators, it is possible to break into existing systemsthat lack interactivity with replay attacks, the disclosed technologyintroduces two way communications, including sensing and effecting, forrandom, unpredictable interactive sessions for authentication. Forexample, the disclosed technology can measure a common factor, such aspulse, with a fingerprint device and another separate biometric device.Each measures the pulse during their main authentication process, whichis combined with, for example, fingerprint recognition, irisrecognition, facial recognition, or retina recognition.

Example embodiments include a fingerprint sensor, an actuator, and aprocessor configured to authenticate a finger based on an image of thefinger generated by the fingerprint sensor and a detected response tothe energy delivered by the actuator. The fingerprint sensor canimplement the actuator in certain embodiments. Various embodiments willnow be described with reference to the drawings.

FIG. 69 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a light source 560 actuator belowan ultrasound transducer array 400, which is below glass 410 with areceiving surface upon which a finger can be placed. In an embodiment,the light source 560 is a component of an optical system 550 asdescribed above with respect to FIG. 68.

FIG. 70 illustrates the embodiment of FIG. 69, in which the light source560 actuator transmits light through the ultrasound transducer array 400and glass 410 to a finger 105 on the receiving surface of the glass 410.The light source 560 emits light at a wavelength, for a duration, and ata power level sufficient to heat at least a portion of the finger 105 onthe receiving surface from temperature T to temperature T+ΔT, where Tmay be close to room temperature and/or finger temperature, and ΔT issufficient to be detectable by the biometric source scanner. The changein temperature ΔT can be discernable by a person, but not so large as toburn the person's finger. For example, ΔT may vary from a tenth of adegree to several degrees on the Fahrenheit scale. The change intemperature ΔT can be detected by the ultrasound transducer array 400 incertain applications. The change in temperature ΔT can be detected by alight sensor of an optical system that includes the light source 560 insome instances.

From the temperature difference ΔT, a specific heat of the tissue of thefinger 105 can be detected. The change in temperature or specific heatcan be different for a live finger and from other objects that could beused in place of a live finger, such as a fake finger or non-livefinger. A processor can authenticate the finger 105 based on whether thetemperature difference ΔT is consistent with an expected temperaturedifference associated with live tissue. The processor can also use animage of the finger 105 generated using the ultrasound transducer array400 to authenticate the finger 105.

An ultrasound transducer array can be used as an actuator and a sensor.FIGS. 71 and 72 illustrate embodiments in which an ultrasound transducerarray is used as a sensor and an actuator. In these embodiments, theultrasound transducer array can deliver energy to an object and alsodetect a response to the energy delivered to the object. While theultrasound transducer array can detect a change in temperature inresponse to ultrasonic heating in FIGS. 71 and 72, an ultrasonictransducer array can detect a variety of other responses to appliedultrasound energy such as responses to pressure. Moreover, theultrasound transducer arrays of FIGS. 71 and 72 can also be used inauthenticating a fingerprint.

FIG. 71 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a point focus ultrasound heater710 that focuses ultrasound from the ultrasound transducer array 400through glass 410 to a point (e.g., a relatively small region, dot, orpixel) of a finger on the receiving surface of the glass 410, such thatthe point focused ultrasound energy incrementally heats the finger. Thepoint focus ultrasound heater 710 can input excitation to oppositeelectrodes that are 180° out of phase and transmit the focusedultrasound energy on each set of electrodes. A change in temperature ofthe finger in the area being heated can be detected using the ultrasoundtransducer array 400 can be used to authenticate the finger.

FIG. 72 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a line focus ultrasound heater 720that focuses ultrasound from the transducer array 400 through glass 410to a line (linear region) of a finger on the receiving surface of theglass 410, such that the line focused ultrasound energy incrementallyheats the finger. The line focus ultrasound heater 720 inputsexcitations on one side of the array. A change in temperature of thefinger in the linear region being heated can be detected using theultrasound transducer array 400 can be used to authenticate the finger.

Other actuators can be integrated with a fingerprint sensor, such as anultrasonic fingerprint sensor. For example, a resistance-basedtemperature sensor can apply resistance-based heating to a finger andsense the change in temperature. FIGS. 73 to 75 illustrate an exampleembodiment of an interactive biometric touch scanner with a fingerprintsensor and integrated with an actuator.

FIG. 73 illustrates an example embodiment of a biometric touch scannerwith two way communication, including a resistance based heater 730capable of sending current through the electrodes (top and bottom metalelectrodes) of the ultrasound transducer array 400 through glass 410 toa finger on the receiving surface of the glass to create a heatingsensation at the user's fingertip.

FIG. 74 illustrates heating a finger 105 with the biometric touchscanner of FIG. 73, The resistance based heater 730 can heat the finger730 to raise a temperature of some or all of the finger 105 fromtemperature T to temperature T+ΔT. In some instances, T may berelatively close to room temperature and ΔT is sufficient to bediscernable by a person, but not so large as to burn the person'sfinger. The resistance can alternatively or additionally be implementedusing electrodes that are separate from the transducer deviceelectrodes.

FIG. 75 illustrates operation of the embodiment of FIGS. 73 and 74. Inthe left portion of FIG. 75, no current is flowing and the finger on thereceiving surface of the finger on the receiving surface is at atemperature T. The temperature sensor can detect temperature T in thisstate. In the right portion of FIG. 75, current is flowing through theelectrodes, generating resistance-based heating, emitting heat throughthe glass to a finger on the glass, and raising the heat of the fingeron the receiving surface to T+ΔT. The temperature sensor can detecttemperature T+ΔT in this state. A processor can use this change intemperature to authentic the finger 105. A processor can also use animage of the finger 105 generated using the transducer array 400 in theauthentication.

Interactive biometric authentication can be performed in a wirelesscommunication device, such as a mobile phone. Wireless communicationdevices include one or more antennas to wirelessly transmit and/orreceive signals. Mobile phones typically include a display, such asliquid crystal display (LCD) or an organic light emitting diode (OLED)display. Fingerprint sensors disclosed herein can be positioned belowthe display of a mobile phone. A mobile phone can also include anengineered glass having surface configured to receive a finger and/orbelow the surface configured to receive the finger. Fingerprint sensorsdisclosed herein can be positioned below the engineered glass in amobile phone. The mobile phones shown in FIGS. 76-81 each include one ormore antennas, a display, and a fingerprint sensor. Any of these mobilephones can include a fingerprint sensor integrated with an opticalsystem in accordance with any suitable principles and advantagesdisclosed herein.

FIGS. 76-81 illustrate representative operations of two waycommunication scenarios associated with voluntary user response inresponse to energy delivered by an actuator. In certain instances,authentication that involves voluntary action can engage a brain of theuser and provide robust authentication that can be even more difficultto fool than involuntary responses in certain applications. A firstexample scenario is illustrated in FIGS. 76-79. A second examplescenario is illustrated in FIGS. 76, 77, 80 and 81.

FIG. 76 illustrates the user interface of an example portablecommunications device including a biometric touch scanner and a displayfor measurements or indications of heart rate, pulse oxidation levels,blood flow, temperature, two way authentication, and fingerprintdetection. In a first operation of the two way communication scenario ofFIGS. 76-81, the device prompts the user to scan a finger.

FIG. 77 illustrates an intermediate operation of the two waycommunication scenarios of FIGS. 76-81, in which the user scans thefinger. The biometric touch scanner extracts biometric information anddisplays biometric information. For example, in FIG. 77 the biometricinformation includes the fingerprint, a heart rate of 100 bpm, a pulseoxidation measurement of 98%, a blood flow estimate of 5 cm/s, and atemperature of 37° C. Any suitable biometric information can bepresented to the user such as any combination of the illustratedbiological information and/or other suitable biological information. Aplacement indictor, such as a green oval, provides feedback if thefingertip is placed properly on the sensor and the biometric informationis acquired. A template image of the scanned fingerprint may bedisplayed within the green oval. This template image signals the userthat her finger is properly placed and that a fingerprint was acquired,without necessarily displaying the fingerprint itself.

FIG. 78 illustrates an intermediate operation of the two waycommunication scenario of FIGS. 76-79. After scanning the biometricinformation, the device generates a sensation at the user's fingertipwith an actuator, by heating, radiation, pressure, neuro-stimulation orany other suitable technique. The user is then prompted to provide aninput corresponding to the sensation that is felt. The sensation maycorrespond to a pulse count, the location at which the sensation isgenerated, the direction of sensation, the shape of the sensationprovided to the finger, the like, or any suitable combination thereof.In FIG. 78, a sensation corresponding to shape A is drawn on the user'sfingertip. The display prompts the user to enter the shape correspondingto the sensation felt by the user, as an interactive form of two wayauthentication. The actuation can be determined such that the two wayauthentication entry is not predictable in advance.

FIG. 79 illustrates another operation of the two way communicationscenario of FIGS. 76-79. In response to the prompt of FIG. 78, the userenters the shape sensed at the fingertip, in this example an A. If theuser enters a shape that matches the actuated shape that was applied tothe finger, the user is authenticated.

FIG. 80 illustrates an intermediate operation of the two waycommunication scenario of FIGS. 76, 77, 80 and 81. After scanning thebiometric measures, the device generates a sensation at the user'sfingertip with an actuator, by heating, radiation, pressure,neuro-stimulation or any other suitable technique. The user is thenprompted to input what sensation is felt. The sensation may correspondto a pulse count, the location or corner at which the sensation isgenerated, the direction of the ultrasound beam (such as top to bottom),or the shape that is drawn on the finger. In FIG. 80, a sensationcorresponding to three pulses is applied to the user's fingertip. Thedisplay prompts the user to enter the number of sensed pulses. Theactuation can be determined such that the two way authentication entryis not predictable in advance.

FIG. 81 illustrates an operation of the two way communication scenarioof FIGS. 76, 77, 80 and 81. In response to the prompt of FIG. 80, theuser enters an indication of the number of pulses felt by the user athis fingertip. If the user enters the correct number of pulses (i.e.,three, in this example), the user is authenticated.

These two scenarios are representative of two way, three way, or higherlevels of multiway authentication. By combining identifying biometricaspects of the user, a fingerprint, and a detected response toactuation, stronger forms of authentication are possible thanauthentication with system in which biometric measures are not used.

FIG. 82 illustrates two way communication scenarios to determine whethera finger exhibits properties of being attached to a live person. A livefinger can provide biometric measurements such as heart rate, bloodflow, temperature, peripheral blood oxygen content, etc. A certainnumber of these biological measurements can be used to authenticate afinger. For instance, it is unlikely that a fake finger can have atleast three of the aforementioned biological measurements that mimic alive finger. While it may be possible simulate one or more of thesemeasures with either an artificial finger, digital simulation, or anon-live finger, using a combination of measures for authentication cansignificantly increase the chances of false identification.

FIG. 82 illustrates the system's ability to determine whether or not afinger is alive. Live finger display 7650 includes biometricmeasurements of heart rate, pulse oximetry (SpO₂), blood flow, andtemperature, all of which are within normal ranges. Live finger display7650 also includes a touch pressure and the user's fingerprint. Incomparison, the fake finger display 7660 does not register the biometricfeatures. Attempts to fool a system by using a non-live finger would notbe successful because the heart rate and other biometric features areunlikely to be within normal limits, even if the fingerprint appears tobe accurate. Artificial fingers that include an expected temperature andvalid appearing fingerprint may or may not be able to simulate all ofthe biometric features. Moreover, they may have particular difficultyresponding to prompts regarding, for example a sensed shape or number ofpulses, as described in the scenarios of FIGS. 76-81. Responding toprompts that are generated in a manner that exhibits statisticalrandomness can be particularly difficult for artificial fingers. Such aprompt can be generated randomly by an algorithm.

FIG. 83 illustrates an example biometric sensing device 8300, with asurface 8305 configured to receive a finger, a fingerprint sensor 8310,and an optical system 550. The fingerprint sensor 8310 generates dataindicative of an image of at least a portion of a fingerprint of thefinger in contact with the surface. The optical system 550, integratedwith the fingerprint sensor 8310, is configured to transmit light to thesurface through the fingerprint sensor. The biometric sensing device8300 can implement any suitable combination of features of the biometricsensors with integrated optical systems discussed herein. The biometricsensing device 8300 can implement one or more features of interactivebiometric scanners discussed herein.

FIG. 84 illustrates an example biometric sensing device 8400, with asurface 8305 configured to receive a finger, ultrasound transducers 405,an optical system 550 and one or more processors 8410. The ultrasonictransducers 405 are configured to transmit an ultrasound signal to thesurface. The optical system 550 is integrated with the ultrasonictransducers 405. The optical system 550 is configured to transmit lightto the surface and receive light reflected from the finger in contactwith the surface. One or more processors 8410 are configured to generatean image of at least a portion of a fingerprint based on a reflection ofthe ultrasound signal from the finger. The one or more processors 8410are configured to generate a liveness parameter based on the receivedlight. The biometric sensing device 8400 can implement any suitablecombination of features of the ultrasound transducers with integratedoptical systems discussed herein. The biometric sensing device 8400 canimplement one or more features of interactive biometric scannersdiscussed herein.

FIG. 85 is a flowchart of a method 8500 of authenticating a user. Inblock 8510, method 8500 transmits, by a fingerprint sensor, a signal tothe finger. In block 8520, method 8500 generates an image of at least aportion of the finger based on a received signal associated with thesignal transmitted to the finger. In block 8530, method 8500 transmitslight through the fingerprint sensor to the finger. In block 8540,method 8500 generates a liveness parameter based on a reflection of thelight from the finger. In block 8550, method 8500 authenticates a userbased on the image and the liveness parameter. The method 8500 can beperformed using a system with one or more features of the biometricsensors with integrated optical systems discussed herein. Moreover, anysuitable features of the interactive two-way communication discussedherein can be performed with the method 8500.

FIG. 86 illustrates an example interactive biometric sensing system8600, with a sensor 8610, an actuator 8620 and a processor 8630. Sensor8610 is configured to generate a biometric image associated with anobject. The biometric image can be an image of at least a portion of afinger. Such an image can be of a surface of a finger or an internalstructure of the finger. Alternatively, the biometric image can be of atleast a portion of a face or an iris. Actuator 8620 is configured todeliver energy to the object. Processor 8630 is configured toauthenticate the object based on the biometric image and a response tothe energy delivered by the actuator.

FIG. 87 illustrates an example interactive biometric sensing device8700, with a surface 8305 configured to receive an object, and a sensor8710. Surface 8305 is configured to receive an object. Sensor 8710 isconfigured to generate biometric information associated with the objectwhile the object is on the surface, delivery energy to the object, anddetect a response to the delivered energy. Accordingly, the sensor 8710functions as both a sensor and an actuator.

FIG. 88 is a flowchart of a method 8800 of authenticating a user. Inblock 8810, method 8800 transmits, by a fingerprint sensor, a signal toa surface configured to receive a finger. In block 8820, method 8800generates an image of at least a portion of the finger based on areceived signal associated with the signal transmitted to the finger. Inblock 8830, method 8800 delivers energy to the finger. In block 8840,method 8800 generates a liveness parameter based on a detected responseto the energy delivered to the finger. In block 8850, method 8800authenticates a user based on the image and the liveness parameter. Anysuitable features of the interactive two-way communication discussedherein can be performed with the method 8800.

FIGS. 89 and 90 illustrate example embodiments with a fingerprint sensorthat is at least partially transparent and that passes light for anoptical system. The fingerprint sensor can include an ultrasoundtransducer array that is at least partially transparent. The ultrasoundtransducer array may be configured as an optically transparentfingerprint sensor. The ultrasound transducer array may be formedentirely, substantially entirely, or only partially from transparentmaterials.

Arrangements in which an optical system and an ultrasonic fingerprintscanner are disposed together (e.g., as in the arrangements of FIGS.81-91) may beneficially prevent spoofing of the authentication process.In particular, such an arrangement may be relatively immune from a highquality dummy fingerprint, as such as dummy fingerprint would notregister as a live finger according to readings from the optical system,which may be associated with greater depths of the finger or objectbeing imaged. In such embodiments, the system may be able to identifyspoof attempts, even if a fraudster were to place a live finger on someother spot on the device.

FIG. 89 illustrates an example embodiment of a biometric sensing device8900 in which light source 560 for an optical system (such as opticalsystem 550) is disposed below an ultrasound transducer array 400, whilean optical sensor 570 for the optical system is disposed above theultrasound transducer array 400. The light source 560 may be an infraredand/or a visible light source, such as an LED or an OLED. The lightsource 560 can be arranged to transmit light at two or more differentwavelengths. The ultrasound transducer array 400 may be configured as afingerprint sensor and may be at least partially transparent to thelight from the light source 560. The optical sensor 570 may be aphotodetector sensitive to infrared and/or visible light and may also beat least partially acoustically transparent (e.g., so as to enableoperation of the fingerprint sensor 400) and at least partiallyoptically transparent (e.g., to as to enable light from light source 560to pass through sensor 570 on its way from the light source 560 to thefinger 105). The photodetector 570 can include a surface to receive thefinger 105 in the device 8900.

As illustrated, the device 8900 of FIG. 89 can include a transparentsubstrate 8910 between the light detector 570 and the ultrasoundtransducer array 400. The substrate 8910 may be formed from glass 410 orany other suitable transparent material (e.g., any material havingsuitable optical and acoustic properties to enable operation of theultrasound transducer array 400 and the optical system including thelight source 560).

FIG. 90 illustrates an example embodiment of a biometric sensing device9000 in which an optical system 550, which can include a light source560 and an optical sensor 570, is disposed below the ultrasoundtransducer array 400. In the example of FIG. 90, the optical system 550need not be acoustically transparent.

FIG. 91 illustrates an example embodiment of a biometric sensing device9100 in which optical system 550 is integrated with and disposed abovethe ultrasound transducer array 400. In the example of FIG. 90, theoptical system 550 may be at least partially acoustically transparent,to enable the ultrasound transducer array 400 to image or scan thefinger 105 through the optical system 550. In the example of FIG. 91,the ultrasound transducer array 400 need not be optically transparent.

FIG. 92 illustrates an example embodiment of a biometric sensing device9200 including ultrasound transducer array 400 and optical systems 550 aand 550 b. One optical system 550 b is disposed laterally from thetransducer array 400 within a device substrate 410. The transducer array400 is disposed between the other optical system 550 a and a surface ofthe biometric sensing device 9200 configured to receive a finger. Theoptical system 550 b in FIG. 92, which is disposed away from thefingerprint sensor, can scan finger 105 and detect veins within thefinger 105 and/or one or more other biometric parameters. The pattern ofveins may be used for authenticating and/or identifying the user. Theoptical system 550 b may include components such as an infrared (IR)light source, an LED, and one or more photoreceptors.

FIGS. 93 and 94 illustrate example embodiments including ultrasoundtransducer array 400 and optical systems 550 a and 550 b and thatincludes at least some components located on an external device.Embodiments such as those illustrated in FIGS. 93 and 94 may provide foran additional or alternative means of authentication or livenessverification, without a user's fingerprint being shared and without theuser having to expose their fingerprint to another potentially untrusteddevice. As examples, the external device may identify the user and/orconfirm liveness using machine learning-derives identificationparameters or using vein or artery patterns in the user's finger orother body part. Additionally, the components on device 9200 andexternal device 590 (as in FIG. 93) or 595 (as in FIG. 94) may worktogether to image a pattern of veins within the finger 105, as part ofauthenticating the finger 105 and associated user.

In the example of FIG. 93, the optical system 550 b includes a lightsource 560 and the external device includes an optical sensor 590. Theoptical sensor 590 may image or scan the finger 105 using visible and/orinfrared light emitted by the light source 560.

In the example of FIG. 94, the optical system 550 b may include a lightsource 560 and may also include an optical sensor 570. The externaldevice may include an external optical system 595 including a lightsource and/or an optical sensor. With the arrangement of FIG. 94, thefinger 105 can be scanned individually by optical sensor 570 and lightsource 560, scanned individually by external optical system 595, orscanned using a combination of these systems. As a first example, theexternal optical system 595 may emit light that is received by opticalsensor 570 to image the finger 105. As a second example, the lightsource 560 may emit light that is received by the external opticalsystem 595 to image the finger 105.

The examples of FIGS. 93 and 94 may include an absorptive pulse oximeteror reflective oximeter. In particular, the external device may receive,with sensor 590, visible and/or IR light, emitted by light source 560,and may measure one or more biometric parameters, such as pulse rateand/or blood oxygen content (e.g., SbO₂), of the finger 105. In someother embodiments, the external device may emit visible and/or IR lightand an optical sensor 570 in the disclosed device may receive the light.In still other embodiments, the external device and/or the discloseddevice may operate independently as reflective oximeters.

Smart Cards

As shown in FIG. 95, a smart card 9500 may be provided that includes anoptically-transparent ultrasonic fingerprint scanner (such as theultrasound transducer array 400 described herein) and with an integratedoptical system (such as the optical system 550 disclosed herein) that isdisposed below the fingerprint scanner. The smart card 9500 canimplement any suitable features of interactive biometric authenticationdisclosed herein by itself and/or in combination with an external device(e.g., a card reader).

The smart card 9500 may be any suitable card, such as a card used forpayment purposes and/or for other purposes. As examples, the smart card9500 may be a credit card, a debit card, a membership card, a rewardscard, an identification card, a security card, a clearance card, asecurity card, an access card, a medical card, an insurance card, etc.Smart card 9500 includes a card body 9502. The card body 9502 can have asize suitable to fit in a wallet. The smart card 9500 can have athickness in a range from 400 μm to 1000 μm. For example, the smart card9500 can have a thickness of about 760 μm. The smart card 9500 can beapproximately 85.60 mm by 53.98 mm. The smart card 9500 can have roundedcorners in certain instances. Additional details and examples of smartcards including ultrasound fingerprint sensors are also described in PCTPatent Application No. PCT/US2018/029309, which is incorporated byreference herein in its entirety.

The ultrasonic fingerprint scanner in the smart card 9500 may be used toauthenticate users presenting the smart card 9500 for authentication.The fingerprint scanner 400 may serve to make it less likely that a cardcould be used by an unauthorized person. As an example, a user maydesire to purchase goods using a smart card 9500 and a payment systemmay be configured to authorize the purchase only upon contemporaneousdetection of the user's fingerprint using the fingerprint scanner 400.

In some embodiments, the optical system 550 may also be used inauthenticating users presenting the smart card 9500 for authentication.As described in further detail herein, optical systems such as opticalsystem 550 may be used to confirm identity and/or liveness independentlyor in collaboration with a fingerprint sensor such as ultrasoundtransducer array 400. The optical system 550 may sense a pulse withinfinger 105 (thus confirming liveness), may sense an oxygen level ofblood in the finger 105 (e.g., using principals similar to those of areflective oximeter, and potentially confirming liveness), may image apattern of veins in the finger 105 (thus confirming identity and/orpreventing at least some forms of potential deception), etc. These aremerely illustrative examples and other examples described herein of howoptical systems may work together with ultrasound systems inauthenticating a user and preventing fraud may also be applied toapplications in smart cards such as the example of FIG. 95.

In certain embodiments, smart card 9500 can include circuitry 9510. Thecircuitry 9510 can perform one or more of the following functions:assists in the detection of the user's fingerprint, assists with thedetection of authentication and/or liveness parameters using the opticalsystem 550, stores the user's fingerprint and/or authentication orliveness information used with the optical system 550 (in a securemanner), or assists in the operation of the smart card 9500 and/or thefingerprint scanners, the optical system 550, the like, or any suitablecombination thereof. The circuitry 9510 may include a power source suchas a photovoltaic cell, a battery, a capacitor, an RF harvestingcircuit, etc. In certain instances, the circuitry 9510 can include oneor more of a smartcard chip, a processor, a memory, a power regulatorcircuit, or the like.

In various embodiments, the smart card 9500 may include one or morecontacts 9512 that connect with an external device 9516 over one or moreelectrical paths 9514. As an example, contacts 9512 may engage with acard reader 9516 when smart card 9500 is inserted into or otherwise incommunication with the card reader 9516. In such embodiments, signalsmay be routed between the card reader 9516 and the fingerprint scanner400 and/or the circuitry 9510. These signals can include power signalsfor powering the fingerprint scanner 400 and/or the circuitry 9510 andcan include fingerprint scans (e.g., where the user's fingerprint isstored remotely), verification results (e.g., where the user'sfingerprint is stored locally on the smart card 9500), the like, or anysuitable combination thereof. In some embodiments, some or all of thetransmission and readout circuitry for fingerprint scanner 400 may beomitted from the card 9500 and provided within external circuitry of thecard reader 9516. This can reduce the cost and complexity of the smartcard 9500.

In some other embodiments, the smart card 9500 may include wirelesscommunication circuitry (including an antenna) in the circuitry 9510 andmay convey data associated with scans of the user's fingerprint and/orfingerprint verification results wirelessly to external circuitry suchas card reader 9516. As examples, the circuits 9510 may transmit signalsusing near-field frequencies or other radio frequency signals.

As illustrated in FIG. 95, the ultrasound scanner 400 and the opticalsystem 550 can be embedded within a card body 9502 of the smart card9500. In some embodiments, the ultrasound scanner 400 and the opticalsystem 550 can be embedded flush with a surface that receives the user'sfinger 105 as shown in FIG. 95. In some other embodiments, theultrasound scanner 400 and the optical system 550 can be flush with asurface opposite to a surface that receives the user's finger 105 andultrasonic waves 9506 may be transmitted by scanner 400 through the cardbody 9502. In other various embodiments, the ultrasound scanner 400 andthe optical system 550 can be embedded within the volume of smart card9500 (e.g., not flush with any surface of the smart card 9500) and canbe completely or nearly completely surrounded by the material formingthe card body 9502. If desired, the optical system 550 may also bedisposed apart from the ultrasound scanner 400, such as in the examplesof FIGS. 92-95. In still other embodiments, the ultrasonic fingerprintscanner 400 and/or some or all of the optical system 550 can be providedon a surface of the smart card 9500 (e.g., on an opposite side or thesame side as the side that receives the user's finger 105).

The ultrasound transducer array 400 may be formed from a piezoelectricmaterial such as a piezoelectric polymer polyvinylidene fluoride (PVDF),a zinc oxide (ZnO) thin film, or other desired materials. The ultrasoundtransducer array 400 of the smart card 9500 can be at least partlyoptically transparent such that light from the optical system canpropagate though the transducer array to the finger 105 and lightreflected from the finger 105 can propagate through the ultrasoundtransducer array 400 to the optical system. A ZnO thin film and/orassociated metal electrodes can be optically transparent. A PVDFpiezoelectric layer can be optically transparent. For instance, asufficiently thin (e.g., less than 9 microns thick) PVDF layer can beoptically transparent. In some embodiments, the card body 9502 may beflexible. In some other embodiments the card body 9502 may be rigid. Theultrasound transducer array 400 can be flexible. For instance, a PVDFbased ultrasound transducer array 400 can be flexible.

Accordingly, a flexible ultrasonic fingerprint scanner that is at leastpartly optically transparent is provided with integrated optics. In thiscase, the ultrasonic fingerprint scanner can be made sufficientlytransparent for light from a light source to propagate therethrough andfor reflected light to propagate though to a photoreceptor. Thisfingerprint scanner with integrated optics can be included in a smartcard.

As shown in FIG. 96, a smart card 9600 can have one or more lightsources such as LEDs 9602 disposed above the ultrasonic fingerprintscanner 400. Such an arrangement can facilitate emitting relativelylarge amounts of light into the finger 105, as light from the LEDs 9602need not pass through the ultrasound transducer array 400. Theillustrated ultrasound transducer array 400 can be transparent or onlybe partially optically transparent and thus may absorb at least somelight passing through it. Such large amounts of light may be beneficialwhen detecting vein patterns in the finger 105 and/or or when measuringpulse or blood oxygen levels in the finger 105.

In another embodiment, a card similar to the illustrated smart card 9600can be implemented without the illustrated optical system 550 disposedbelow the ultrasonic fingerprint scanner 400. In such an embodiment, theultrasonic fingerprint scanner 400 can be non-optically transparent oroptically transparent.

Mobile Devices

As shown in FIG. 97, a mobile device 9700 may be provided that includesan optically-transparent ultrasonic fingerprint scanner (such as theultrasound transducer array 400 described herein) and with an opticalsystem (such as the optical system 550 disclosed herein) that isdisposed below the fingerprint scanner. The mobile device 9700 may be amobile phone such as a smart phone, a tablet device, a portable device,a handheld device, etc. The mobile device 9700 may also include adisplay 9702, which may be a touchscreen display, and one or moreantennas such as antenna 9704. The antenna 9704 can transmit and/orreceive radio frequency signals. The mobile device 9704 can implementany suitable features of interactive biometric authentication disclosedherein.

FIG. 97 illustrates that ultrasound transducer array 400 and opticalsystem 550 may be disposed on a front side of the mobile device 9700.The array 400 and optical system 550 may alternatively or additionallybe disposed on a rear side or lateral side face of the mobile device9700. While FIG. 97 illustrates the array 400 and optical system 550 asbeing below the display 9702, this is merely illustrative. In general,array 400 and optical system 550 may be disposed to any side of thedisplay 9702 or can be disposed within the display 9702 (e.g., behindthe display 9702). Optical system 550 is shown as being smaller than thearray 400 in FIG. 97 merely for illustrative purposes. In general,optical system 550 may be smaller, larger, or the same size as theultrasound transducer array 400.

Multiple Device Authentication

FIG. 98 illustrates multiple device authentication using a user's device9800 and a confirming device 9810. The user's device 9800 and theconfirming device 9810 can be mobile phones, for example. The user'sdevice 9800 includes an integrated fingerprint and light scanner 9802and a light scanner 9804. The integrated fingerprint and light scanner9802 can include an ultrasound fingerprint sensor and a light scanner inaccordance with any suitable principles and advantages disclosed herein.For example, the ultrasound fingerprint sensor can be at least partiallytransparent and the light scanner can transmit and receive light throughthe ultrasound fingerprint sensor. The light scanner 9804 can detect oneor more biometric parameters, such as a PPG waveform, a heart rate, arespiration rate, a vein pattern, the like, or any combination thereof.The light scanner 9804 can include a reflexive oximeter. The confirmingdevice 9810 includes an integrated fingerprint and light scanner 9812and a light scanner 9814.

In some instances, readings of reflective pulse oximeters may beadversely affected by differences in ambient light and/or pressure oftissue on the sensor. Accordingly, it may be beneficial to configure thelight scanner 9804 of the user's device 9800 and configure the lightscanner 9814 of the confirming device 9810 to reduce and/or minimizedifferences in ambient light during scanning and/or to reduce and/orminimize any difference in pressures between scanned tissue and thesensors. As an example, the light scanners 9804 and 9814 may be locatedin similar positions and/or with similar orientations on theirrespective devices, such that a user holding the devices generallypresses a scanned finger onto each of the light scanners 9804 and 9814with the same finger, at a similar level of pressure, with a similarorientation, the like, or any combination thereof. This can reduceand/or minimize differences in scans obtained by the two light scanners.

A user can be authenticated using his or her fingerprint using thefingerprint sensor of the integrated fingerprint and light scanner 9802.A confirmation of the match can be sent to indicate that the user'sfingerprint has been authenticated. This confirmation can be sent to abank or other interested party. The user, in the same location as thefingerprint sensor, has PPG waveforms read and analyzed. Data, such as apattern associated with the PPG waveform and/or one or more otherbiometric parameters can be sent to the interested party. The user, witha second finger on a confirming device, such as another phone orcomputer, can have a biometric parameter sensed using a light scanner9814. The light scanner 9814 can have substantially the same or similarequipment and software to determine a biometric parameter (e.g., take aPPG waveform reading). Associated data is sent to the interested party(e.g., a bank) to confirm a match with data from the user's phone.

The interested party through our device's software and machine learningtechniques can detect features that relate to age and state of healththat correspond to the user over past interactions. For example, partsof a PPG waveform reading can be different, as there is variability hourto hour. This variability can help prevent a replay attack. Since theuser has two fingers on both devices, any hour to hour differences inthe PPG waves are shown to be the same at the time on both devices.

Accordingly, a biometric parameter or pattern (e.g., a PPG scan) can beused for identification and/or authentication purposes. Substantiallythe same or similar equipment and software on two different devices canbe used in the authentication. The ultrasound fingerprint scan can bedone only on one of the devices at the same time and location as thelight scan on the other device. The other device does not detect thefingerprint, so the user's fingerprint will not be exposed to the otherdevice or copied.

Machine learning can be used to develop an algorithm to add identifyingcharacteristics available for a second step authentication. Thosecharacteristics can show up on both phones and they could be storedexternal to the authentication devices, such as at a bank or cloudwebsite for confirmations. The characteristics could relate to, forexample, a heart condition or age or more specific readings of heartrate than just the summary pulse number. A scammer using his or her ownfinger for a PPG scan along with a replay attack of a recordedultrasound fingerprint authentication of the user they are hacking,assuming they could break into a phone to get the ultrasound fingerprintalgorithm, would have to have similar characteristics (e.g., be aboutthe same age and heart condition and breathing condition) as theauthentic user to be authenticated with this technology.

Another algorithm could capture some of the relevant PPG waves that arevariable hour to hour, so that there would also be an authenticationelement that picks up the various of the PPG reading, so that each timethe variability would make the reading slightly different, though thevariability would be recorded as the same for the user's finger on theuser device 9800 and the user's finger on the confirming device 9810.

Using a second device for authentication allows for second stepauthentication by a second device without exposing the user'sfingerprint to the second device. This can be useful for a credit cardcompany or merchant authenticating a translation. A driver of aridesharing service or a hotel manager may already have a knownlocation, and their device can be used for authenticating a user withoutreading the user's fingerprint. With this technology, the authenticatingparty, such as a bank, could then know that the user has traveled fromPalo Alto to San Diego and their location has been verified by a globalpositioning system (GPS) on both the smart phones of either a Uberdriver or the person at a San Diego hotel front desk. The same livesigns from the reflective oximeter appear on the user's phone where hisor her fingerprint is authenticated at the same time as the life sign isbeing read. This can provide robust authentication.

Dynamic Biometrics and Machine Learning

Although some embodiments are discussed with reference to a livenessparameter, any suitable biometric sensing device disclosed herein can beused to detect dynamic biometrics. Dynamic biometrics can represent apattern of one or more biometric parameters over time. Detecting dynamicbiometrics can involve one or more of imaging, measuring, or analyzingreal-time physiological responses of living tissue to external and/orinternal stimuli. By using dynamic biometrics, a biometricauthentication system can be more resistant to presentation attacks.Moreover, dynamic biometrics can be more distinctive than staticreadings, such as an average pulse or blood oxygen level. In fact,dynamic biometric can be sufficiently distinct to verify a person incombination with a fingerprint. One or more processors (e.g., one ormore of the processors 1065, 8410, or 8630) can track a one or morebiometric parameters generated using any suitable biometric sensingdevice disclosed herein over time. As an example, biorhythms related toa reflective oximeter can be used in authentication.

The dynamic biometrics can be detecting during a fingerprintauthentication processor. One or more processors can apply machinelearning to identify traits of a person responding to prompts from anactuator of any suitable interactive biometric sensing device disclosedherein. Any suitable interactive device or interactive system disclosedherein that is able to actuate (for example, apply haptic energy) canenable measurement of individual characteristics as to how a personresponds to our device during an authentication session. Theprocessor(s) can detect patterns when a person responds to a prompt,such as being prompted to stand up while being authenticated by both anultrasound sensor and our optical system that is generating aphotoplethysmogram (PPG).

Dynamic biometrics can be used to generate improved, personalizedbiometric readings. Machine learning can be used to find patterns invarious biometric parameters, such as PPG scans, respiration rate, heartrate, the like, or any combination thereof. Such patterns can be used inbiometric authentication. For instance, PPG waves can change hour tohour, based on the activity of a person. If a mismatch from what isexpected by machine learning and/or artificial intelligence is detectedby PPG readings, this can cause authentication to fail or lead to anauthentication system taking additional readings to authenticate aperson.

Machine learning and/or other techniques can capture a reasonablydistinct signature of a person from one or more biometric parameters(e.g., a PPG) during authentication (e.g., while generating an image ofat least a portion of a finger). The reasonably distinct signature canbe verified using two different sensing devices, such as sensing deviceson different mobile phones. As an example, a first mobile phone can reada biometric parameter using a reflective oximeter of an optical systembelow an ultrasound fingerprint sensor. On a second device, the personcan use a reflective oximeter of the second device to match thebiometric parameter without risking exposure of sensitive data, such asa fingerprint. Having the substantially the same reflective oximeter,substantially same frequency, and substantially the same software canaid in verifying the biometric parameter for robust authentication.

A biometric pattern is not permanent and can change hour to hour basedon a person's activity, yet still provide distinctive aspects that canappear day to day. Accordingly, a way to duplicate some life pattern,that is not unique but still reasonably distinctive (e.g., about 1 inseveral thousand) can be implemented on two devices concurrently. Sincethe readings should be slightly different, the user is not giving awayhis or her unique fingerprint or facial identification while undergoingan additional measurement for authentication.

Additional Embodiments

Embodiments of the disclosed technology can use an ultrasound transducerarray in accordance with any suitable principles and advantagesdiscussed herein as part of an interaction with a user beingauthenticated by a fingerprint scanner and/or a camera scanner equippedwith facial recognition and motion-sensing software at the same time.Having some action(s) by the user triggered and then detected by the twodevices can provide increased security. Incorporating several differentinteractive efforts with two biometric devices create multiplepossibilities that would be hard for a scammer to fake and/or anticipatewith a ‘replay’ attack. The disclosed technology can employ two or moremodalities in such interactions that authenticate the user, while theyare also registering that an action has been taken in response to atrigger by one of the authentication devices.

For example, if the user were triggered by the ultrasound transducerarray 400 to make a specific motion, such as tilting his or her faceupwards, in response to haptic energy applied to the tip of the finger,or alternatively tilting the face down, in response to haptic energyapplied to the bottom of the fingerprint, such a motion could bedetected by a portable computing device equipped with a threedimensional motion sensor, that, for example combines a camera andassociated infrared beam, at the same time that a facial identificationauthentication is completed.

In this case, a user's finger, whose fingerprint is being authenticatedby the ultrasound system, can detect a signal of haptic energy from thefingerprint device to prompt the user to react. Next, the user's face,authenticated by a separate device such as a phone camera, is detectedto have moved. For example, part of the authentication scheme can promptthe user to close an eye or wink. Both the facial authentication andfacial movement information are then passed to a system to register andconfirm the actions as part of the authentication process. In this case,one would have created cross-talk and work being done, coordinated andconfirmed between the two different biometric devices.

An advantage of registering movement by a user's face, instead of hisarm, has the advantage that the user's face can be confirmed to be fromthe same person as the confirmed fingerprint.

In an embodiment, the order can be reversed: A camera/flash/illuminatorcould also initiate an action, such as a small red light, prompting theuser to temporarily remove their fingerprint that is being authenticatedfrom the screen. At the same time, the red light might also signal thatthe user briefly move their head from side-to-side. In this case, thecamera/flash/illuminator can trigger an action that is read as aresponse by both devices. The camera/flash/illuminator registers thehead movement and the fingerprint reader registers the removal of thefinger.

With interactivity, multiple scans can occur concurrently during anauthentication session. In such scans, several different bio-metricmeasurements can be combined, with a common measurement/factor and withinteractions between different bio-metric devices. A system can includedifferent biometric sensors and a fingerprint sensor/device equippedwith multi-mode biometric scanning.

An interactive biometric system can measure a common factor, such aspulse, with a fingerprint device and another separate biometric device,with each measuring the same common factor during their mainauthentication process while they each are measuring two other factors,such as the fingerprint and face.

The disclosed technology enables confirmation of aliveness of afingertip or other appendage from another biometric scanning device onthe same phone, computer, or other device. Examples of detecting analiveness parameter include scanning a pulse from minute color changesin the face, from the movement of blood via heart pulses, from thephone's camera as well as from our fingerprint device, using the similarprincipal of reflective oximeter. Both of these pulse scans can happenwhile the user's facial identification is confirmed by the phone'scamera and the fingerprint scanner is confirming the user's fingerprint.This can create a common denominator of a measurement shared andconfirmed by itself and the camera authenticating the face or iris.

If both the camera and our fingerprint scanner are measuring the pulse,then the pulse would increase if a user stood up, and that would bemeasured by both the camera and the fingerprint scanner—so such aninteractive action would be measured by two different biometric sensorson the phone. Furthermore, a camera device with a 3D motion sensor coulddetect such a motion change from the user standing up while the camerais also reading a change in the pulse.

In order to confirm that the fingerprint scanner heart beat is similarto measurement by the camera's facial identification scanner, the heartrate or timing between beats can be compared to determine that they arethe same, even though the beats occur at different times in the fingerand in the face due to different distances from the heart.

Combining two or more different biometric scanning devices with multiplevariables or measurements (for example, heart rates measured bydifferent sensor modalities) at the same time that the two multi-modebiometric scanning devices are authenticating that the user is the sameperson with a recognized face and fingerprint, it is harder for ascammer to fool both sensors, as a fake face would have to register thesame pulse as that of a fake fingerprint.

Other sensor modalities can be included, such as measuring respirationrates by camera, which can be similar to using an optical system 550 forreflective oximetry. For example, plethysmograms captured by the opticalsystem can be analyzed using image processing and pattern recognitiontechniques to determine changes in respiration rates.

In some embodiments, multi-factor authentication can be performed on oneintegrated ultrasound/reflectance pulse oximeter device and somemulti-factor authentication can be prompted by the larger device (suchas a smart phone). As an example, a smart phone can vibrate as a prompt.

A device, such as a mobile phone, can program at what stage an alivenessreading is involved in authentication and/or at what stage aninteractive authentication session is implemented. Users can desire thattheir mobile phones to turn on reliably and quickly, and therefore abasic ultrasound fingerprint scan can unlock the phone. Such ameasurement can reduce the chances of a false rejection. Built into amobile phone's settings or in an application installed on the phone, auser could configure a setting so that authentication can also includeanother biometric reading (e.g., pulse, temperature and/or SPO2 reading)for access to certain files and/or a user could add an interactiveauthentication with random prompts coming from the phone computing unitor an application specific integrated circuit. A user, or a third partywanting to authenticate the user, could program the phone, as to whenthe device would employ interactive authentication to add securityaccess and identity protection for external connections, such as signingon to the office remotely or the cloud or for shopping or bank sites.Accordingly, a biometric authentication system can operate in differentmodes with different levels of authentication. Such a system can beconfigurable such that accessing certain files and/or certainfunctionalities of a device can involve higher levels of authentication,such as any suitable features of the interactive biometric sensingdiscussed herein.

To implement interactive authentication in accordance with theprinciples and advantages discussed herein, an application on a mobilephone or computer can accept input from a third party to initiate anactuator prompting a user. Accordingly, a third party can direct randomand/or unpredictable interactive biometric prompts and/or biometricmeasurements.

An interactive biometric sensing system can include a sensor configuredto generate a biometric image associated with an object, a promptingdevice configured to prompt an action associated with the object, and aprocessor configured to authenticate the object based on the biometricimage and a detected biometric response to the prompt. For instance, thesensor can be a fingerprint sensor. The prompting device can prompt theuser to take action, such as by providing text and/or audio to promptthe user to take action (e.g., to stand up, to jump, etc.) that shouldresult in a detection biometric response. Then the biometric responsecan be detected and the processor can authenticate the used using boththe biometric image and the detected response.

One aspect of the disclosed technology is an acoustic fingerprintsensing device. The device includes an array of ultrasonic transducersconfigured to transmit an ultrasound signal having a frequency in arange from 50 megahertz (MHz) to 500 MHz. The ultrasonic transducersinclude a piezoelectric film. The device further includes first metalelectrodes. The device further includes second metal electrodes that canbe orthogonal to the first metal electrodes. The first metal electrodesand the second metal electrodes enable addressing of the ultrasonictransducers of the array. The device further includes a surfaceconfigured to receive a finger. The device further includes a processorconfigured to generate an image of at least a portion of a fingerprintof the finger in contact with the surface based on a reflection of theultrasound signal from the finger. The device further includes anactuator configured to vary the temperature of and/or the pressure onthe finger in contact with the surface.

In an embodiment, the actuator includes the ultrasonic transducers.

In an embodiment the actuator varies the pressure in a series of pulsesfocused on a configurable region of the receiving surface. In anembodiment, the actuator focuses heat on a configurable region of thereceiving surface to increase the temperature at the region by at least0.1° C.

In an embodiment, the frequency of the ultrasound signal is in a rangefrom 125 MHz to 250 MHz. In an embodiment, the frequency of theultrasound signal is in a range from 50 MHz to 100 MHz. In anembodiment, the piezoelectric film has a thickness in a range from 3micrometers (μm) to 75 μm. In an embodiment, the piezoelectric film hasa thickness in a range from 10 micrometers (μm) to 20 μm.

In an embodiment, the device further includes a receiver circuitconfigured to process an electronic receive signal generated by thearray of ultrasonic transducers in response to the reflection to providea processed signal to the processor.

In an embodiment, the image has a resolution of at least 500 pixels perinch.

In an embodiment, the first metal electrodes are in physical contactwith a plate that includes the surface.

In an embodiment, the piezoelectric film comprises at least one of zincoxide, aluminum nitride, or lead zirconium titanate. In an embodiment,the surface is a surface of a plate that comprises glass and a matchinglayer, and the matching layer has a thickness corresponding to a quarterof a wavelength of the ultrasound signal in material of the matchinglayer.

In an embodiment, the processor is configured to estimate a force atwhich the finger contacts the receiving surface based on an area of thefinger in contact with the receiving surface. In an embodiment, theprocessor is configured to detect a temperature of the finger based on asound speed associated with the reflection. In an embodiment, theprocessor is configured to detect a parameter associated with a livenessof the finger based on the reflection, and to provide an indication ofwhether the finger is part of a live human based on the livenessparameter.

Another aspect is a method of authenticating a fingerprint. The methodincludes: addressing ultrasound transducers of an array of ultrasoundtransducers using first metal electrodes and second metal electrodes,the second metal electrodes orthogonal to the first metal electrodes.The method further includes transmitting, by the array of ultrasonictransducers, a first ultrasound signal in a frequency range from 50megahertz (MHz) to 500 MHz towards a receiving surface. The methodfurther includes receiving, using the one or more ultrasonictransducers, a reflection of the first ultrasound signal. The methodfurther includes generating a first image of at least a portion of afinger on the receiving surface based on the reflection of the firstultrasound signal. The method further includes varying, by an actuator,the temperature of and/or the pressure on the finger in contact with thereceiving surface. The method further includes transmitting, by thearray of ultrasound transducers, a second ultrasound signal having afrequency in a range from 50 megahertz (MHz) to 500 MHz. The methodfurther includes generating a second image of at least a second portionof the fingerprint of the finger based on a reflection of the secondultrasound signal. The method further includes authenticating thefingerprint in response to a comparison of the first image and thesecond image, corresponding to the change in temperature of and/or thepressure on the finger in contact with the receiving surface.

In an embodiment, the method further includes varying, by the actuator,the pressure in a series of pulses focused on a configurable region ofthe receiving surface. In embodiment, the method further includesheating, by the actuator, a configurable region of the receiving surfaceto increase the temperature at the region. In some instances, thetemperature can be increased by at least 0.1° C.

In an embodiment, the method further includes processing an electronicreceive signal generated by the array of ultrasonic transducers inresponse to the reflection of the first ultrasound signal.

In an embodiment, the first metal electrodes are in physical contactwith a plate that includes the receiving surface.

In an embodiment, the method further includes estimating a force atwhich the finger contacts the receiving surface based on an area of thefinger in contact with the receiving surface. In an embodiment, themethod further includes detecting a temperature of the finger based on asound speed associated with the reflection. In an embodiment, the methodfurther includes detecting a parameter associated with a liveness of thefinger based on the reflection. In an embodiment, the method furtherincludes providing an indication of whether the finger is part of a livehuman based on the liveness parameter.

Another aspect is a biometric sensing and actuating device forfingerprint identification. The device includes an array of ultrasonictransducers configured to transmit an ultrasound signal having afrequency in a range from 50 megahertz (MHz) to 500 MHz. The devicefurther includes first metal electrodes. The device further includessecond metal electrodes that can be orthogonal to the first metalelectrodes. The first metal electrodes and the second metal electrodesenable addressing of the ultrasonic transducers of the array. The devicefurther includes a surface configured to receive a finger. The devicefurther includes a processor configured to generate an image of at leasta portion of a fingerprint of the finger based on a reflection of theultrasound signal from the finger. The device further includes anactuator that varies the temperature of and/or the pressure on thefinger in contact with the receiving surface. In an embodiment, theactuator comprises the ultrasonic transducer. In an embodiment, theactuator varies the pressure in a series of pulses focused on a regionof the receiving surface. In an embodiment, the actuator focuses heat ona region of the receiving surface. This can increase the temperature atthe region by a detectable amount, such as at least 0.1° C.

Another aspect is a method for authenticating a fingerprint using abiometric sensing and actuating device. The method includes addressingultrasound transducers of an array of ultrasound transducers using firstmetal electrodes and second metal electrodes, the second metalelectrodes that can be orthogonal to the first metal electrodes. Themethod further includes transmitting, by the array of ultrasonictransducers, a first ultrasound signal having a frequency in a rangefrom 50 megahertz (MHz) to 500 MHz. The method further includesgenerating a first image of at least a portion of a first fingerprint ofa first finger based on a reflection of the first ultrasound signal. Themethod further includes heating a region of a receiving surface toincrease a temperature of the region by least 0.1° C. The method furtherincludes transmitting, by the array of ultrasound transducers, a secondultrasound signal having a frequency in a range from 50 megahertz (MHz)to 500 MHz. The method further includes detecting the increase intemperature based on a reflection of the second ultrasound signal. Themethod further includes authenticating the first fingerprint based onthe first image and the detected increase in temperature.

Another aspect is a method for authenticating a fingerprint using abiometric sensing and actuating device. The method includes addressingultrasound transducers of an array of ultrasound transducers using firstmetal electrodes and second metal electrodes, the second metalelectrodes that can be orthogonal to the first metal electrodes. Themethod further includes transmitting, by the array of ultrasonictransducers, a first ultrasound signal having a frequency in a rangefrom 50 megahertz (MHz) to 500 MHz. The method further includesgenerating a first image of at least a portion of a first fingerprint ofa first finger based on a reflection of the first ultrasound signal. Themethod further includes varying the pressure of a region of a receivingsurface on a finger in contact with the receiving surface by xx N/m² (oryy psi). The method further includes transmitting, by the array ofultrasound transducers, a second ultrasound signal having a frequency ina range from 50 megahertz (MHz) to 500 MHz. The method further includesgenerating a second image of at least a second portion of a secondfingerprint of a second finger based on a reflection of the secondultrasound signal. The method further includes authenticating the firstfingerprint in response to a comparison of the first image and thesecond image, corresponding to the change in pressure of the region ofthe receiving surface. The discernable pressure change can be set by themechanical noise floor in the transducer, the transducer sensitivity,the noise figure of the electronics, or any combination thereof.

Another aspect is a biometric sensing device for fingerprintidentification using both optical and ultrasonic signals. The deviceincludes a light source configured to transmit light. The light can havea frequency in a range from 400 nm to 1000 nm, for example. The deviceincludes an array of ultrasonic transducers configured to transmit anultrasound signal having a frequency in a range from 50 megahertz (MHz)to 500 MHz. The device further includes first metal electrodes. Thedevice further includes second metal electrodes that can be orthogonalto the first metal electrodes. The first metal electrodes and the secondmetal electrodes enable addressing of the ultrasonic transducers of thearray. The device further includes a surface configured to receive afinger. The device further includes a processor configured toauthenticate the finger based on a reflection of the light and/or theultrasound image from the finger.

In an embodiment, the light source transmits light through theultrasonic transducer, the first metal electrodes, and the second metalelectrodes, wherein the ultrasonic transducer is at least partiallytransparent, wherein the first metal electrodes are at least partiallytransparent, and wherein the second metal electrodes are at leastpartially transparent. In this embodiment, the array of ultrasonictransducers are between the optical emitter and the receiving surface,and the ultrasonic transducers, first metal electrodes, and second metalelectrodes are at least partially transparent to the transmitted light.

In an embodiment, the ultrasound transducers transmit the ultrasoundsignal through the light source. In this embodiment, the ultrasoundsignal passes through the light source. This embodiment may include anarray of light sources in squares on a plane, each square bounded byprojections of the first metal electrodes and second metal electrodes inthe plane.

In an embodiment, the light source adjoins the ultrasound transducer. Inthis embodiment, the light source and the ultrasound transducer are sideby side but in close proximity to each other.

APPLICATIONS AND CONCLUSION

Some of the embodiments described above have provided examples inconnection with ultrasound-based fingerprint sensors. However, theprinciples and advantages of the embodiments can be used for any othersuitable devices, systems, apparatuses, and/or methods that couldbenefit from such principles and advantages. Although described in thecontext of fingerprints, one or more features described herein can alsobe utilized in detecting any other suitable part of a human or animal.

The various features and processes described herein may be implementedindependently of one another, or may be combined in various ways. Allpossible combinations and sub combinations are intended to fall withinthe scope of this disclosure. In addition, certain methods or processblocks may be omitted in some implementations. The methods and processesdisclosed herein are also not limited to any particular sequence, andthe blocks or states relating thereto can be performed in any othersequences that are appropriate. For example, described blocks or statesmay be performed in an order other than that specifically disclosed, ormultiple blocks or states may be combined in a single block or state.The example blocks or states may be performed in serial, in parallel, orin some other manner as appropriate. Blocks or states may be added to orremoved from the disclosed example embodiments as suitable. The examplesystems and components described herein may be configured differentlythan described. For example, elements may be added to, removed from, orrearranged compared to the disclosed example embodiments. Variousembodiments can apply different techniques for fabricating differenttypes of electronic devices.

Aspects of this disclosure can be implemented in various devices. Forexample, the acoustic biometric sensing devices discussed herein can beimplemented in a mobile phone such as a smart phone, a tablet computer,a steering wheel, a gun, a door, a door handle, a wall, an elevator, orany other suitable application that could benefit from any of theprinciples and advantages discussed herein.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the disclosure. Indeed, the novel devices, systems, apparatus,methods, and systems described herein may be embodied in a variety ofother forms; furthermore, various omissions, substitutions and changesin the form of the methods and systems described herein may be madewithout departing from the spirit of the disclosure. For example, whileblocks are presented in a given arrangement, alternative embodiments mayperform similar functionalities with different components and/or circuittopologies, and some blocks may be deleted, moved, added, subdivided,combined, and/or modified. Each of these blocks may be implemented in avariety of different ways. Any suitable combination of the elements andacts of the various embodiments described above can be combined toprovide further embodiments.

1. (canceled)
 2. A method of interactively authenticating a person, themethod comprising: transmitting, by a fingerprint sensor, a signal to afinger of the person; generating an image of at least a portion of thefinger based on a received signal associated with the signal transmittedto the finger; delivering energy to the finger; detecting a response tothe energy delivered to the finger, wherein the response is indicativeof whether the person is alive; and authenticating the person based on(i) processing the image and (ii) separately processing an indication ofthe response to the energy delivered to the finger.
 3. The method ofclaim 2, wherein the fingerprint sensor comprises an ultrasoundtransducer, and the delivering energy is performed using the ultrasoundtransducer.
 4. The method of claim 2, wherein the fingerprint sensorcomprises an ultrasound transducer that is at least partiallytransparent, and the delivering energy is performed using a light sourcearranged to transmit light to the finger through the ultrasoundtransducer.
 5. The method of claim 4, wherein the detecting is performedusing a light detector arranged to receive a reflection of the lightfrom the finger.
 6. The method of claim 2, wherein the delivering theenergy is performed using a resistance based heater.
 7. The method ofclaim 2, wherein the processing the indication of the response comprisesdetecting a change in a liveness parameter associated with the fingerresulting from the delivering the energy.
 8. The method of claim 2,wherein the processing the indication of the response comprisesdetecting a change in temperature of at least part of the fingerresulting from the delivering the energy.
 9. The method of claim 2,wherein the delivering the energy is performed by the fingerprintsensor.
 10. The method of claim 2, wherein the delivering the energy isperformed by an actuator that is separate from the fingerprint sensor.11. The method of claim 2, wherein the finger is positioned on a surfaceduring both the transmitting and the delivering energy.
 12. The methodof claim 2, wherein the response involves voluntary action by theperson.
 13. The method of claim 2, wherein the detecting comprisesdetecting the response via a user interface.
 14. The method of claim 2,wherein the method is performed by a mobile phone.
 15. The method ofclaim 2, wherein the method is performed on a millisecond timescale. 16.The method of claim 2, wherein the delivering the energy is performedunpredictably.
 17. The method of claim 2, wherein the delivering theenergy is associated with an expected response from the person, andwherein the authenticating is based on the response to the energydelivered to the finger corresponding to the expected response.
 18. Amobile device with interactive biometric authentication, the mobiledevice comprising: an antenna configured to a transmit a wirelesscommunication signal; a surface configured to receive a finger; a sensorconfigured to provide biometric image data associated with the fingerbeing positioned on the surface; and a processor in communication withthe sensor, the processor configured to authenticate the finger based onthe biometric image data and also based on a response to energydelivered to the finger, wherein the response to the energy delivered tothe finger indicates whether the finger is part of a person who isalive, and wherein an indication of the response is obtained separatelyfrom the biometric image data.
 19. The mobile device of claim 18,wherein the response to the energy delivered to the finger involvesvoluntary action by the person.
 20. The mobile device of claim 18,wherein the response to the energy delivered to the finger includes achange in a liveness parameter associated with the finger.
 21. Themobile device of claim 18, wherein the indication of the response isobtained by the mobile device at a different time than the biometricimage data.
 22. The mobile device of claim 18, wherein the sensor isconfigured to deliver the energy to the finger.
 23. The mobile device ofclaim 18, wherein an actuator is configured to deliver the energy to thefinger, and the actuator includes hardware separate from the sensor. 24.The mobile device of claim 18, further comprising a light source,wherein the sensor comprises ultrasound transducers that are at leastpartially transparent, and wherein the light source is configured todeliver the energy to the finger through the ultrasound transducers. 25.The mobile device of claim 18, wherein the sensor comprises ultrasoundtransducers.
 26. The mobile device of claim 18, further comprisingengineered glass positioned between the sensor and the surface, theengineered glass being scratch and damage resistant.
 27. The mobiledevice of claim 18, wherein the mobile device is arranged as a mobilephone.
 28. An interactive biometric authentication system comprising:means for obtaining a fingerprint image of at least a portion of afinger; means for delivering energy to the finger to cause a change in aliveness parameter associated with the finger, the change in theliveness parameter indicating whether the finger is of a live person;and a processor in communication with both the means for obtaining thefingerprint image and the means for delivering energy, the processorconfigured to authenticate the live person based on (i) the fingerprintimage and (ii) an indication of the change in the liveness parametercaused by the means for delivering energy.
 29. An interactive biometricauthentication system comprising: means for obtaining a fingerprintimage of at least a portion of a finger of a person; means fordelivering energy to the finger of the person; and a processor incommunication with both the means for obtaining the fingerprint imageand the means for delivering energy, the processor configured toauthenticate the person based on (i) the fingerprint image and (ii) anindication of a voluntary action of the person in response to energybeing delivered to the finger by the means for delivering energy.